* 3.9.3: Oops running xfstests [not found] <510292845.4997401.1369279175460.JavaMail.root@redhat.com> @ 2013-05-23 3:21 ` CAI Qian 2013-05-23 3:51 ` Dave Chinner 0 siblings, 1 reply; 18+ messages in thread From: CAI Qian @ 2013-05-23 3:21 UTC (permalink / raw) To: xfs; +Cc: stable Fedora-19 based distro and LVM partitions. [ 301.385639] XFS (dm-2): Mounting Filesystem [ 301.434479] XFS (dm-2): Ending clean mount [ 301.972477] XFS (dm-0): Mounting Filesystem [ 302.264395] XFS (dm-0): Ending clean mount [ 302.819270] BUG: unable to handle kernel paging request at ffff8801fbf19000 [ 302.858096] IP: [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 302.886832] PGD 1ddf067 PUD 20bdf9067 PMD 1fb92e063 PTE 80000001fbf19161 [ 302.923831] Oops: 0003 [#1] SMP [ 302.939820] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg iTCO_wdt e1000e iTCO_vendor_support ixgbe mdio dca ptp pps_core pcspkr hpwdt hpilo lpc_ich microcode mfd_core xfs libcrc32c sd_mod crc_t10dif ata_generic pata_acpi mgag200 i2c_algo_bit drm_kms_helper ata_piix ttm drm libata i2c_core hpsa dm_mirror dm_region_hash dm_log dm_mod [ 303.231016] CPU 3 [ 303.240132] Pid: 19129, comm: attr Not tainted 3.9.3 #1 HP ProLiant DL120 G7 [ 303.278111] RIP: 0010:[<ffffffff813022fa>] [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 303.316747] RSP: 0018:ffff8801d1bd5ad8 EFLAGS: 00010286 [ 303.343135] RAX: ffff8801fbeb0020 RBX: ffff8801fd7e0000 RCX: 000000000000d00b [ 303.381919] RDX: fffffffffff96fd8 RSI: ffff8801fbf193f8 RDI: ffff8801fbf19000 [ 303.420250] RBP: ffff8801d1bd5b30 R08: 7720657079742065 R09: 6c75646f6d202b50 [ 303.459969] R10: 465320646574726f R11: 707075736e75206e R12: ffff8801fbeb0000 [ 303.498346] R13: 00000000000003d8 R14: 0000000000001800 R15: 000000000000007b [ 303.535369] FS: 00007f0026971740(0000) GS:ffff88020b460000(0000) knlGS:0000000000000000 [ 303.576159] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 303.604432] CR2: ffff8801fbf19000 CR3: 00000001e6403000 CR4: 00000000000407e0 [ 303.640327] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 303.675585] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 303.711631] Process attr (pid: 19129, threadinfo ffff8801d1bd4000, task ffff8801e676b580) [ 303.753011] Stack: [ 303.762850] ffffffffa01971a1 0000000000000000 0000000000000000 ffff8801fbeb0020 [ 303.799908] ffff8801fbeb0000 0000000000000018 ffff8801fbeb0000 0000000000000000 [ 303.836554] ffff8801e2066618 ffff8801d3ac2680 ffff8801d16d9a20 ffff8801d1bd5b68 [ 303.874096] Call Trace: [ 303.886242] [<ffffffffa01971a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [ 303.925533] [<ffffffffa0197467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [ 303.962494] [<ffffffffa0198a2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [ 303.996629] [<ffffffffa0194850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [ 304.033242] [<ffffffffa01a2d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [ 304.069208] [<ffffffffa019578c>] xfs_attr_set_int+0x30c/0x420 [xfs] [ 304.100911] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [ 304.126845] [<ffffffffa0195d1f>] xfs_attr_set+0x7f/0x90 [xfs] [ 304.156492] [<ffffffffa018ba12>] xfs_xattr_set+0x42/0x70 [xfs] [ 304.185868] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [ 304.214985] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [ 304.245755] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [ 304.273386] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [ 304.299844] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [ 304.329476] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [ 304.357290] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [ 304.382481] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [ 304.412902] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [ 304.442884] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [ 304.469523] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 304.502054] Code: 00 00 48 81 fa a8 02 00 00 72 05 40 38 fe 74 41 48 83 ea 20 48 83 ea 20 4c 8b 1e 4c 8b 56 08 4c 8b 4e 10 4c 8b 46 18 48 8d 76 20 <4c> 89 1f 4c 89 57 08 4c 89 4f 10 4c 89 47 18 48 8d 7f 20 73 d4 [ 304.605187] RIP [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 304.631499] RSP <ffff8801d1bd5ad8> [ 304.649339] CR2: ffff8801fbf19000 [ 304.665667] ---[ end trace b3069cd0e14b6ad0 ]--- [ 304.665670] general protection fault: 0000 [#2] SMP [ 304.665682] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg iTCO_wdt e1000e iTCO_vendor_support ixgbe mdio dca ptp pps_core pcspkr hpwdt hpilo lpc_ich microcode mfd_core xfs libcrc32c sd_mod crc_t10dif ata_generic pata_acpi mgag200 i2c_algo_bit drm_kms_helper ata_piix ttm drm libata i2c_core hpsa dm_mirror dm_region_hash dm_log dm_mod [ 304.665684] CPU 2 [ 304.665684] Pid: 1202, comm: rhts-test-runne Tainted: G D 3.9.3 #1 HP ProLiant DL120 G7 [ 304.665687] RIP: 0010:[<ffffffff812960e2>] [<ffffffff812960e2>] avc_has_perm_noaudit+0x72/0x110 [ 304.665688] RSP: 0018:ffff8801dee91cd8 EFLAGS: 00010202 [ 304.665688] RAX: 00010a102504ff45 RBX: 0000000000000143 RCX: ffff8801f9a82df0 [ 304.665689] RDX: ffff8801f9a82e10 RSI: 0000000000000004 RDI: 0000000000000143 [ 304.665689] RBP: ffff8801dee91d10 R08: 0000000000000000 R09: ffff8801dee91d34 [ 304.665690] R10: 0000000000000001 R11: ffff880200937800 R12: 0000000000000143 [ 304.665690] R13: 0000000000200000 R14: 0000000000000000 R15: 0000000000000004 [ 304.665691] FS: 00007f7f2c2b7740(0000) GS:ffff88020b440000(0000) knlGS:0000000000000000 [ 304.665692] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 304.665692] CR2: 0000000001d5e0a8 CR3: 00000001def1c000 CR4: 00000000000407e0 [ 304.665693] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 304.665694] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 304.665694] Process rhts-test-runne (pid: 1202, threadinfo ffff8801dee90000, task ffff8801e00a5040) [ 304.665694] Stack: [ 304.665696] ffff8801dee91da8 0000000000000296 0000000000200000 0000000000000143 [ 304.665697] 0000000000000000 0000000000000004 ffff8801fb965a00 ffff8801dee91d90 [ 304.665698] ffffffff8129966c ffff880200000000 ffffffff8105e848 000000d000000002 [ 304.665698] Call Trace: [ 304.665701] [<ffffffff8129966c>] cred_has_capability+0x5c/0x100 [ 304.665702] [<ffffffff8105e848>] ? dup_mm+0x208/0x660 [ 304.665705] [<ffffffff81178819>] ? alloc_pages_current+0xa9/0x170 [ 304.665706] [<ffffffff8129973e>] selinux_capable+0x2e/0x40 [ 304.665707] [<ffffffff81299885>] selinux_vm_enough_memory+0x35/0x50 [ 304.665710] [<ffffffff81293a83>] security_vm_enough_memory_mm+0x13/0x20 [ 304.665710] [<ffffffff8105e82f>] dup_mm+0x1ef/0x660 [ 304.665712] [<ffffffff8105f6a9>] copy_process.part.26+0x9d9/0x13d0 [ 304.665713] [<ffffffff8106019d>] do_fork+0xad/0x340 [ 304.665714] [<ffffffff811b7f40>] ? get_unused_fd_flags+0x30/0x40 [ 304.665715] [<ffffffff810604b6>] sys_clone+0x16/0x20 [ 304.665717] [<ffffffff816196b9>] stub_clone+0x69/0x90 [ 304.665718] [<ffffffff81619359>] ? system_call_fastpath+0x16/0x1b [ 304.665727] Code: db 81 31 c0 65 ff 04 25 00 0a 01 00 48 8d 4a e0 48 85 d2 48 0f 45 c1 eb 0d 48 8b 40 20 48 85 c0 74 3f 48 83 e8 20 48 85 c0 74 36 <3b> 18 75 ea 66 3b 70 08 75 e4 44 3b 60 04 75 de 48 8d 50 0c 48 [ 304.665728] RIP [<ffffffff812960e2>] avc_has_perm_noaudit+0x72/0x110 [ 304.665729] RSP <ffff8801dee91cd8> [ 304.665763] ---[ end trace b3069cd0e14b6ad1 ]--- [ 304.898489] ============================================================================= [ 304.898489] BUG kmalloc-4096 (Tainted: G D ): Padding overwritten. 0xffff8801fbeb7c28-0xffff8801fbeb7fff [ 304.898490] ----------------------------------------------------------------------------- [ 304.898490] [ 304.898491] INFO: Slab 0xffffea0007efac00 objects=7 used=7 fp=0x (null) flags=0x20000000004080 [ 304.898492] Pid: 357, comm: systemd-udevd Tainted: G B D 3.9.3 #1 [ 304.898492] Call Trace: [ 304.898495] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 [ 304.898497] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [ 304.898498] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 [ 304.898500] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 [ 304.898501] [<ffffffff811820e3>] check_slab+0x73/0x100 [ 304.898503] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 [ 304.898504] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 [ 304.898506] [<ffffffff81161b57>] ? vma_link+0xb7/0xc0 [ 304.898508] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 [ 304.898509] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 [ 304.898510] [<ffffffff811bda63>] seq_read+0x2e3/0x3b0 [ 304.898512] [<ffffffff8119c56c>] vfs_read+0x9c/0x170 [ 304.898513] [<ffffffff8119c939>] sys_read+0x49/0xa0 [ 304.898514] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b CAI Qian _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-23 3:21 ` 3.9.3: Oops running xfstests CAI Qian @ 2013-05-23 3:51 ` Dave Chinner 2013-05-24 8:52 ` CAI Qian 0 siblings, 1 reply; 18+ messages in thread From: Dave Chinner @ 2013-05-23 3:51 UTC (permalink / raw) To: CAI Qian; +Cc: stable, xfs On Wed, May 22, 2013 at 11:21:17PM -0400, CAI Qian wrote: > Fedora-19 based distro and LVM partitions. Cai: As I've asked previously please include all the relevant information about your test system and the workload it is running when the problem occurs. Stack traces aren't any good to us in isolation, and just dumping them on us causes unnecessary round trips. http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F > [ 304.898489] ============================================================================= > [ 304.898489] BUG kmalloc-4096 (Tainted: G D ): Padding overwritten. 0xffff8801fbeb7c28-0xffff8801fbeb7fff > [ 304.898490] ----------------------------------------------------------------------------- > [ 304.898490] > [ 304.898491] INFO: Slab 0xffffea0007efac00 objects=7 used=7 fp=0x (null) flags=0x20000000004080 > [ 304.898492] Pid: 357, comm: systemd-udevd Tainted: G B D 3.9.3 #1 > [ 304.898492] Call Trace: > [ 304.898495] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 > [ 304.898497] [<ffffffff8118176d>] ? init_object+0x3d/0x70 > [ 304.898498] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 > [ 304.898500] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 > [ 304.898501] [<ffffffff811820e3>] check_slab+0x73/0x100 > [ 304.898503] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 > [ 304.898504] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 > [ 304.898506] [<ffffffff81161b57>] ? vma_link+0xb7/0xc0 > [ 304.898508] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 > [ 304.898509] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 > [ 304.898510] [<ffffffff811bda63>] seq_read+0x2e3/0x3b0 > [ 304.898512] [<ffffffff8119c56c>] vfs_read+0x9c/0x170 > [ 304.898513] [<ffffffff8119c939>] sys_read+0x49/0xa0 > [ 304.898514] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b That's something different, and indicates memory corruption is being seen as a result of something that is occuring through the /proc or /sys filesystems. Unrelated to XFS, I think... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-23 3:51 ` Dave Chinner @ 2013-05-24 8:52 ` CAI Qian 2013-05-27 5:36 ` Dave Chinner 0 siblings, 1 reply; 18+ messages in thread From: CAI Qian @ 2013-05-24 8:52 UTC (permalink / raw) To: Dave Chinner; +Cc: stable, xfs ----- Original Message ----- > From: "Dave Chinner" <david@fromorbit.com> > To: "CAI Qian" <caiqian@redhat.com> > Cc: xfs@oss.sgi.com, stable@vger.kernel.org > Sent: Thursday, May 23, 2013 11:51:15 AM > Subject: Re: 3.9.3: Oops running xfstests > > On Wed, May 22, 2013 at 11:21:17PM -0400, CAI Qian wrote: > > Fedora-19 based distro and LVM partitions. > > Cai: As I've asked previously please include all the relevant > information about your test system and the workload it is running > when the problem occurs. Stack traces aren't any good to us in > isolation, and just dumping them on us causes unnecessary round > trips. > > http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F Sometimes, those information is going to drive me crazy due to the amount of information need to gather from a system that been already returned to the automation testing system pool and I never have access to it anymore. Some of the information has like very little percentage of the relevant as far as I can tell. I knew sometimes that 1% percentage does count but the amount of efforts need to gather that 1% just crazy. :) Since we have been in the same company, feel free to ping me and I can give you the instruction to access the system and reproducer for it. Also, I have been reproduced this on several x64 systems and nothing special about the hardware and this panic at memmove is also very much similar to those s390x/ppc64 stack overrun cases which has memove, xfs-leaf on the trace, http://oss.sgi.com/archives/xfs/2013-05/msg00768.html I will provide the information as far I knew for now. - kernel version (uname -a): 3.9.3 - xfsprogs version (xfs_repair -V): Fedora-19 xfsprogs-3.1.10 - number of CPUs: 8 - contents of /proc/meminfo: MemTotal: 16367152 kB MemFree: 15723040 kB Buffers: 1172 kB Cached: 313016 kB SwapCached: 0 kB Active: 252388 kB Inactive: 172832 kB Active(anon): 111376 kB Inactive(anon): 260 kB Active(file): 141012 kB Inactive(file): 172572 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 8257532 kB SwapFree: 8257532 kB Dirty: 5008 kB Writeback: 0 kB AnonPages: 110800 kB Mapped: 22944 kB Shmem: 564 kB Slab: 69100 kB SReclaimable: 26524 kB SUnreclaim: 42576 kB KernelStack: 1488 kB PageTables: 5896 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 16441108 kB Committed_AS: 265500 kB VmallocTotal: 34359738367 kB VmallocUsed: 45288 kB VmallocChunk: 34347010568 kB HardwareCorrupted: 0 kB AnonHugePages: 2048 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 77440 kB DirectMap2M: 16764928 kB - contents of /proc/mounts: nothing special. Just Fedora-19 autopart - contents of /proc/partitions: nothing special. Just Fedora-19 autopart - RAID layout (hardware and/or software): Nothing special, 06:21:51,812 INFO kernel:[ 27.480775] mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 0, phy 0, sas_addr 0x500000e0130ddbe2 06:21:51,812 NOTICE kernel:[ 27.539634] scsi 0:0:0:0: Direct-Access IBM-ESXS MAY2073RC T107 PQ: 0 ANSI: 5 06:21:51,812 INFO kernel:[ 27.592421] mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x500000e0130fa8f2 06:21:51,812 NOTICE kernel:[ 27.651334] scsi 0:0:1:0: Direct-Access IBM-ESXS MAY2073RC T107 PQ: 0 ANSI: 5 06:21:51,812 NOTICE kernel:[ 27.753114] sd 0:0:0:0: [sda] 143374000 512-byte logical blocks: (73.4 GB/68.3 GiB) 06:21:51,812 NOTICE kernel:[ 27.798987] sd 0:0:1:0: [sdb] 143374000 512-byte logical blocks: (73.4 GB/68.3 GiB) 06:21:51,812 NOTICE kernel:[ 27.847388] sd 0:0:0:0: [sda] Write Protect is off 06:21:51,812 NOTICE kernel:[ 27.847396] sd 0:0:1:0: [sdb] Write Protect is off 06:21:51,812 DEBUG kernel:[ 27.847398] sd 0:0:1:0: [sdb] Mode Sense: d7 00 00 08 06:21:51,812 DEBUG kernel:[ 27.904710] sd 0:0:0:0: [sda] Mode Sense: d7 00 00 08 06:21:51,812 NOTICE kernel:[ 27.905323] sd 0:0:1:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA 06:21:51,812 NOTICE kernel:[ 27.960059] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA 06:21:51,812 INFO kernel:[ 27.998854] sdb: sdb1 06:21:51,812 NOTICE kernel:[ 28.025249] sd 0:0:1:0: [sdb] Attached SCSI disk 06:21:51,812 INFO kernel:[ 28.096714] sda: sda1 sda2 06:21:51,812 NOTICE kernel:[ 28.139844] sd 0:0:0:0: [sda] Attached SCSI disk - LVM configuration: nothing special. Just Fedora-19 autopart. The below information from the installation time. Later, everything been formatted to XFS. name = vg_ibmls4102-lv_root status = True kids = 0 id = 7 parents = ['existing 139508MB lvmvg vg_ibmls4102 (3)'] uuid = wVn1JV-DQ4U-vXHD-liJi-kX0M-O6eA-geU4gs size = 51200.0 format = existing ext4 filesystem major = 0 minor = 0 exists = True protected = False sysfs path = /devices/virtual/block/dm-1 partedDevice = parted.Device instance -- model: Linux device-mapper (linear) path: /dev/mapper/vg_ibmls4102-lv_root type: 12 sectorSize: 512 physicalSectorSize: 512 length: 104857600 openCount: 0 readOnly: False externalMode: False dirty: False bootDirty: False host: 13107 did: 13107 busy: False hardwareGeometry: (6527, 255, 63) biosGeometry: (6527, 255, 63) PedDevice: <_ped.Device object at 0x7f5ffd504b00> target size = 51200.0 path = /dev/mapper/vg_ibmls4102-lv_root format args = [] originalFormat = ext4 target = None dmUuid = None VG device = LVMVolumeGroupDevice instance (0x7f5fee7e3590) -- name = vg_ibmls4102 status = True kids = 3 id = 3 parents = ['existing 69505MB partition sda2 (2) with existing lvmpv', 'existing 70005MB partition sdb1 (5) with existing lvmpv'] uuid = X0Bee2-lAuT-egUe-AXc1-a69j-dfmK-3ex1CB size = 139508 format = existing None major = 0 minor = 0 exists = True protected = False sysfs path = partedDevice = None target size = 0 path = /dev/mapper/vg_ibmls4102 format args = [] originalFormat = None target = None dmUuid = None free = 0.0 PE Size = 4.0 PE Count = 34877 PE Free = 0 PV Count = 2 LV Names = ['lv_home', 'lv_root', 'lv_swap'] modified = False extents = 34877.0 free space = 0 free extents = 0.0 reserved percent = 0 reserved space = 0 PVs = ['existing 69505MB partition sda2 (2) with existing lvmpv', 'existing 70005MB partition sdb1 (5) with existing lvmpv'] LVs = ['existing 71028MB lvmlv vg_ibmls4102-lv_home (6) with existing ext4 filesystem', 'existing 51200MB lvmlv vg_ibmls4102-lv_root (7) with existing ext4 filesystem', 'existing 17280MB lvmlv vg_ibmls4102-lv_swap (8) with existing swap'] percent = 0 mirrored = False stripes = 1 snapshot total = 0MB VG space used = 51200MB - type of disks you are using: nothing special - write cache status of drives: missed; need to reprovision the system. - size of BBWC and mode it is running in: missed; need to reprovision the system. - xfs_info output on the filesystem in question: missed; need to reprovision the system. - dmesg output showing all error messages and stack traces: http://people.redhat.com/qcai/stable/console.txt > > > > [ 304.898489] > > ============================================================================= > > [ 304.898489] BUG kmalloc-4096 (Tainted: G D ): Padding > > overwritten. 0xffff8801fbeb7c28-0xffff8801fbeb7fff > > [ 304.898490] > > ----------------------------------------------------------------------------- > > [ 304.898490] > > [ 304.898491] INFO: Slab 0xffffea0007efac00 objects=7 used=7 fp=0x > > (null) flags=0x20000000004080 > > [ 304.898492] Pid: 357, comm: systemd-udevd Tainted: G B D 3.9.3 > > #1 > > [ 304.898492] Call Trace: > > [ 304.898495] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 > > [ 304.898497] [<ffffffff8118176d>] ? init_object+0x3d/0x70 > > [ 304.898498] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 > > [ 304.898500] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 > > [ 304.898501] [<ffffffff811820e3>] check_slab+0x73/0x100 > > [ 304.898503] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 > > [ 304.898504] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 > > [ 304.898506] [<ffffffff81161b57>] ? vma_link+0xb7/0xc0 > > [ 304.898508] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 > > [ 304.898509] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 > > [ 304.898510] [<ffffffff811bda63>] seq_read+0x2e3/0x3b0 > > [ 304.898512] [<ffffffff8119c56c>] vfs_read+0x9c/0x170 > > [ 304.898513] [<ffffffff8119c939>] sys_read+0x49/0xa0 > > [ 304.898514] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b > > That's something different, and indicates memory corruption is being > seen as a result of something that is occuring through the /proc or > /sys filesystems. Unrelated to XFS, I think... > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com > -- > To unsubscribe from this list: send the line "unsubscribe stable" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-24 8:52 ` CAI Qian @ 2013-05-27 5:36 ` Dave Chinner 2013-05-27 6:04 ` CAI Qian ` (2 more replies) 0 siblings, 3 replies; 18+ messages in thread From: Dave Chinner @ 2013-05-27 5:36 UTC (permalink / raw) To: CAI Qian; +Cc: stable, xfs On Fri, May 24, 2013 at 04:52:06AM -0400, CAI Qian wrote: > > > ----- Original Message ----- > > From: "Dave Chinner" <david@fromorbit.com> > > To: "CAI Qian" <caiqian@redhat.com> > > Cc: xfs@oss.sgi.com, stable@vger.kernel.org > > Sent: Thursday, May 23, 2013 11:51:15 AM > > Subject: Re: 3.9.3: Oops running xfstests > > > > On Wed, May 22, 2013 at 11:21:17PM -0400, CAI Qian wrote: > > > Fedora-19 based distro and LVM partitions. > > > > Cai: As I've asked previously please include all the relevant > > information about your test system and the workload it is running > > when the problem occurs. Stack traces aren't any good to us in > > isolation, and just dumping them on us causes unnecessary round > > trips. > > > > http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F > Sometimes, those information is going to drive me crazy due to the > amount of information need to gather from a system that been already > returned to the automation testing system pool and I never have access > to it anymore. So automate the collection of all the static information. > Some of the information has like very little percentage > of the relevant as far as I can tell. I knew sometimes that 1% percentage > does count but the amount of efforts need to gather that 1% just crazy. :) It answers basic questions about your system: - how many CPUs - how much RAM - kernel version, userspace xfs utilities version - mount option - filesystem size - filesystem configuration - storage configuration - storage hardware - error messages that are occurring. Whether you consider that irrelevant information doesn't matter. They fact is that it is basic information that *we need* to understand what your environment is that reproduced the problem. If you don't supply that information, we can't even begin to triage the problem. > Since we have been in the same company, feel free to ping me and I can > give you the instruction to access the system and reproducer for it. Also, I don't scale to manually gathering information from every person that reports a problem. My time is far better spent reviewing information and asking for more than it is gathering information. > I have been reproduced this on several x64 systems and nothing special "nothing special". You say that a lot. You are giving you opinion abou tyour hardware, leaving me with no information to determine whether there is anything special about myself. I don't want your opinion - I want *data*. > I will provide the information as far I knew for now. > - kernel version (uname -a): 3.9.3 > - xfsprogs version (xfs_repair -V): Fedora-19 xfsprogs-3.1.10 > - number of CPUs: 8 What type of CPUs? That's why we ask for /proc/cpuinfo.... > - contents of /proc/mounts: nothing special. Just Fedora-19 autopart What are the contents? > - contents of /proc/partitions: nothing special. Just Fedora-19 autopart Contents, please. > - RAID layout (hardware and/or software): > Nothing special, You say nothing special. I see: > 06:21:51,812 INFO kernel:[ 27.480775] mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 0, phy 0, sas_addr 0x500000e0130ddbe2 > 06:21:51,812 NOTICE kernel:[ 27.539634] scsi 0:0:0:0: Direct-Access IBM-ESXS MAY2073RC T107 PQ: 0 ANSI: 5 > 06:21:51,812 INFO kernel:[ 27.592421] mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x500000e0130fa8f2 > 06:21:51,812 NOTICE kernel:[ 27.651334] scsi 0:0:1:0: Direct-Access IBM-ESXS MAY2073RC T107 PQ: 0 ANSI: 5 Hardware RAID of some kind. So, details, please. Indeed, googling for "IBM-ESXS MAY2073RC" turns up this lovely link: http://webcache.googleusercontent.com/search?client=safari&rls=x86_64&q=cache:dD2J_ZuKGF4J:http://www.ibm.com/support/entry/portal/docdisplay%3Flndocid%3DMIGR-5078767%2BIBM-ESXS+MAY2073RC&oe=UTF-8&redir_esc=&hl=en&ct=clnk "Corrects firmware defect that can cause data corruption" http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5092677 "Additional Information A subset of optional IBM Options hard drive shipped between April 2010 and January 2013 running firmware levels A3C2, A3C0, A3BE, or A3B8 may be exposed to a possible undetected data loss or data error during a proximal write." That might be relevant to a filesystem corruption problem, yes? Start to understand why we ask for basic information about your hardware now? > - LVM configuration: nothing special. Just Fedora-19 autopart. The below information > from the installation time. Later, everything been formatted to XFS. Manually? > name = vg_ibmls4102-lv_root status = True kids = 0 id = 7 > parents = ['existing 139508MB lvmvg vg_ibmls4102 (3)'] > uuid = wVn1JV-DQ4U-vXHD-liJi-kX0M-O6eA-geU4gs size = 51200.0 > format = existing ext4 filesystem > major = 0 minor = 0 exists = True protected = False > sysfs path = /devices/virtual/block/dm-1 partedDevice = parted.Device instance -- > model: Linux device-mapper (linear) path: /dev/mapper/vg_ibmls4102-lv_root type: 12 ..... Ugh. That's unreadable..... > PVs = ['existing 69505MB partition sda2 (2) with existing lvmpv', > 'existing 70005MB partition sdb1 (5) with existing lvmpv'] > LVs = ['existing 71028MB lvmlv vg_ibmls4102-lv_home (6) with existing ext4 filesystem', > 'existing 51200MB lvmlv vg_ibmls4102-lv_root (7) with existing ext4 filesystem', > 'existing 17280MB lvmlv vg_ibmls4102-lv_swap (8) with existing swap'] But what I see here is that there are partitions and LVM, and everything is using ext4. That's not useful to me. What I'm after is the output of pvdisplay, vgdisplay and lvdisplay. That's not very hard... > - type of disks you are using: nothing special Do you really think they are nothing special now after reading the above information? > - write cache status of drives: missed; need to reprovision the system. > - size of BBWC and mode it is running in: missed; need to reprovision the system. > - xfs_info output on the filesystem in question: missed; need to reprovision the system. > - dmesg output showing all error messages and stack traces: > http://people.redhat.com/qcai/stable/console.txt Looking at dmesg output (next time, please, just dmesg) there's been kernel modules bounced in and out of the kernel so all the kernel modules are force loaded. You've run trinity on this system for at least several hours before running other tests, which has left who-knows what mess behind in memory. It's been OOMed at least 15 times and it's been up and running under test workloads for over 24 hours before I see the first XFS filesystem get mounted. The PID that has tripped over the attribute problem is called "comm" which means it is unrelated to xfstests. And in processing the oops, a pair of slab corruptions in the DRM subsystem is detected, which kind of points at large scale memory corruption unrelated to XFS. That's most likely, because I've already pointed out memory corruption way outside XFS is occurring: > > > ============================================================================= > > > [ 304.898489] BUG kmalloc-4096 (Tainted: G D ): Padding > > > overwritten. 0xffff8801fbeb7c28-0xffff8801fbeb7fff > > > [ 304.898490] > > > ----------------------------------------------------------------------------- > > > [ 304.898490] > > > [ 304.898491] INFO: Slab 0xffffea0007efac00 objects=7 used=7 fp=0x > > > (null) flags=0x20000000004080 > > > [ 304.898492] Pid: 357, comm: systemd-udevd Tainted: G B D 3.9.3 > > > #1 > > > [ 304.898492] Call Trace: > > > [ 304.898495] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 > > > [ 304.898497] [<ffffffff8118176d>] ? init_object+0x3d/0x70 > > > [ 304.898498] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 > > > [ 304.898500] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 > > > [ 304.898501] [<ffffffff811820e3>] check_slab+0x73/0x100 > > > [ 304.898503] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 > > > [ 304.898504] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 > > > [ 304.898506] [<ffffffff81161b57>] ? vma_link+0xb7/0xc0 > > > [ 304.898508] [<ffffffff811bda63>] ? seq_read+0x2e3/0x3b0 > > > [ 304.898509] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 > > > [ 304.898510] [<ffffffff811bda63>] seq_read+0x2e3/0x3b0 > > > [ 304.898512] [<ffffffff8119c56c>] vfs_read+0x9c/0x170 > > > [ 304.898513] [<ffffffff8119c939>] sys_read+0x49/0xa0 > > > [ 304.898514] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b > > > > That's something different, and indicates memory corruption is being > > seen as a result of something that is occuring through the /proc or > > /sys filesystems. Unrelated to XFS, I think... So, can you reproduce this problem on a clean, *pristine* system that hasn't been used for destructive testing for 24 hours prior to running xfstests? Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-27 5:36 ` Dave Chinner @ 2013-05-27 6:04 ` CAI Qian 2013-05-28 4:31 ` CAI Qian 2013-05-30 3:45 ` CAI Qian 2 siblings, 0 replies; 18+ messages in thread From: CAI Qian @ 2013-05-27 6:04 UTC (permalink / raw) To: Dave Chinner; +Cc: stable, xfs Hi, > So automate the collection of all the static information. Yes, that probably the best to do. > > I will provide the information as far I knew for now. > > - kernel version (uname -a): 3.9.3 > > - xfsprogs version (xfs_repair -V): Fedora-19 xfsprogs-3.1.10 > > - number of CPUs: 8 > > What type of CPUs? That's why we ask for /proc/cpuinfo.... Reproduced on both, Dual-Core AMD Opteron(tm) Processor 8218 and Intel(R) Xeon(R) CPU E31220 @ 3.10GHz. > > > - RAID layout (hardware and/or software): > > Nothing special, > > You say nothing special. I see: > > > 06:21:51,812 INFO kernel:[ 27.480775] mptsas: ioc0: attaching ssp device: > > fw_channel 0, fw_id 0, phy 0, sas_addr 0x500000e0130ddbe2 > > 06:21:51,812 NOTICE kernel:[ 27.539634] scsi 0:0:0:0: Direct-Access > > IBM-ESXS MAY2073RC T107 PQ: 0 ANSI: 5 > > 06:21:51,812 INFO kernel:[ 27.592421] mptsas: ioc0: attaching ssp device: > > fw_channel 0, fw_id 1, phy 1, sas_addr 0x500000e0130fa8f2 > > 06:21:51,812 NOTICE kernel:[ 27.651334] scsi 0:0:1:0: Direct-Access > > IBM-ESXS MAY2073RC T107 PQ: 0 ANSI: 5 > > Hardware RAID of some kind. So, details, please. Indeed, googling > for "IBM-ESXS MAY2073RC" turns up this lovely link: > > http://webcache.googleusercontent.com/search?client=safari&rls=x86_64&q=cache:dD2J_ZuKGF4J:http://www.ibm.com/support/entry/portal/docdisplay%3Flndocid%3DMIGR-5078767%2BIBM-ESXS+MAY2073RC&oe=UTF-8&redir_esc=&hl=en&ct=clnk > > "Corrects firmware defect that can cause data corruption" > > http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5092677 > > "Additional Information > A subset of optional IBM Options hard drive shipped between April > 2010 and January 2013 running firmware levels A3C2, A3C0, A3BE, or > A3B8 may be exposed to a possible undetected data loss or data error > during a proximal write." > > > That might be relevant to a filesystem corruption problem, yes? > Start to understand why we ask for basic information about your > hardware now? OK, and this was also reproduced on, [ 9.778658] hpsa 0000:04:00.0: hpsa0: <0x323a> at IRQ 43 using DAC [ 9.796557] [TTM] Zone kernel: Available graphics memory: 4066866 kiB [ 9.796558] [TTM] Zone dma32: Available graphics memory: 2097152 kiB [ 9.796558] [TTM] Initializing pool allocator [ 9.796594] [TTM] Initializing DMA pool allocator [ 9.930529] scsi0 : ata_piix [ 9.934804] scsi1 : ata_piix [ 9.937735]SATA max UDMA/133 cmd 0x1400 ctl 0x1408 bmdma 0x1420 irq 17 [ 10.143857] fbcon: mgadrmfb (fb0) is primary device [ 10.143943] ata2: SATA max UDMA/133 cmd 0x1410 ctl 0x1418 bmdma 0x1428 irq 17 [ 10.146870] scsi2 : hpsa [ 10.146900] ata_piix 0000:00:1f.5: MAP [ [ 10.146901] P0 -- P1 -- ] [ 10.149376] scsi3 : ata_piix [ 10.151329] scsi4 : ata_piix [ 10.151702] ata3: SATA max UDMA/133 cmd 0x1440 ctl 0x1448 bmdma 0x1460 irq 17 [ 10.151705] ata4: SATA max UDMA/133 cmd 0x1450 ctl 0x1458 bmdma 0x1468 irq 17 [ 10.152789] hpsa 0000:04:00.0: RAID device c2b3t0l0 added. [ 10.152790] hpsa 0000:04:00.0: Direct-Access device c2b0t0l0 added. [ 10.153206] scsi 2:3:0:0: RAID HP P410 5.14 PQ: 0 ANSI: 5 [ 10.153563] scsi 2:0:0:0: Direct-Access HP LOGICAL VOLUME 5.14 PQ: 0 ANSI: 5 [ 10.213887] [drm] mga base 0 [ 10.285126] Console: switching to colour frame buffer device 128x48 [ 10.387399] mgag200 0000:01:00.1: fb0: mgadrmfb frame buffer device [ 10.387399] mgag200 0000:01:00.1: registered panic notifier [ 10.387441] [drm] Initialized mgag200 1.0.0 20110418 for 0000:01:00.1 on minor 0 [ 10.469431] ata3: SATA link down (SStatus 0 SControl 300) [ 10.508564] ata4: SATA link down (SStatus 0 SControl 300) [ 10.765662] ata2.00: SATA link down (SStatus 0 SControl 300) [ 10.793575] ata2.01: SATA link down (SStatus 0 SControl 300) [ 10.833455] ata1.00: SATA link down (SStatus 0 SControl 300) [ 10.861780] ata1.01: SATA link down (SStatus 0 SControl 300) [ 10.900174] sd 2:0:0:0: [sda] 143305920 512-byte logical blocks: (73.3 GB/68.3 GiB) [ 10.940769] sd 2:0:0:0: [sda] Write Protect is off [ 10.965161] sd 2:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 11.013492] sda: sda1 sda2 [ 11.029787] sd 2:0:0:0: [sda] Attached SCSI disk [ 11.288535] bio: create slab <bio-1> at 1 > > > - LVM configuration: nothing special. Just Fedora-19 autopart. The below > > information > > from the installation time. Later, everything been formatted to XFS. > > Manually? Partitioning and formatting during the installation time using kickstart files. > So, can you reproduce this problem on a clean, *pristine* system > that hasn't been used for destructive testing for 24 hours prior to > running xfstests? Yes, I can try that and let you know if I can. Regards, CAI Qian _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-27 5:36 ` Dave Chinner 2013-05-27 6:04 ` CAI Qian @ 2013-05-28 4:31 ` CAI Qian 2013-05-30 3:45 ` CAI Qian 2 siblings, 0 replies; 18+ messages in thread From: CAI Qian @ 2013-05-28 4:31 UTC (permalink / raw) To: Dave Chinner; +Cc: stable, xfs Below is more information, > What type of CPUs? That's why we ask for /proc/cpuinfo.... + lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 1 Core(s) per socket: 2 Socket(s): 4 NUMA node(s): 4 Vendor ID: AuthenticAMD CPU family: 15 Model: 65 Model name: Dual-Core AMD Opteron(tm) Processor 8218 Stepping: 2 CPU MHz: 2599.855 BogoMIPS: 5199.28 Virtualization: AMD-V L1d cache: 64K L1i cache: 64K L2 cache: 1024K NUMA node0 CPU(s): 0,4 NUMA node1 CPU(s): 1,5 NUMA node2 CPU(s): 2,6 NUMA node3 CPU(s): 3,7 or + lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 42 Model name: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz Stepping: 7 CPU MHz: 3092.721 BogoMIPS: 6185.44 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 8192K NUMA node0 CPU(s): 0-3 > > > - contents of /proc/mounts: nothing special. Just Fedora-19 autopart > > What are the contents? + cat /proc/mounts rootfs / rootfs rw 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=8201464k,nr_inodes=2050366,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs rw,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 /dev/mapper/rhel_ibm--ls41--02-root / xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,relatime 0 0 tmpfs /tmp tmpfs rw,seclabel 0 0 /dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 /dev/mapper/rhel_ibm--ls41--02-home /home xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 or + cat /proc/mounts rootfs / rootfs rw 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4054696k,nr_inodes=1013674,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs rw,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 /dev/mapper/rhel_storageqe--13-root / xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 tmpfs /tmp tmpfs rw,seclabel 0 0 debugfs /sys/kernel/debug debugfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime 0 0 /dev/mapper/rhel_storageqe--13-home /home xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 /dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 > > > - contents of /proc/partitions: nothing special. Just Fedora-19 autopart > > Contents, please. + cat /proc/partitions major minor #blocks name 8 16 71687000 sdb 8 17 71685120 sdb1 8 0 71687000 sda 8 1 512000 sda1 8 2 71173120 sda2 253 0 8257536 dm-0 253 1 52428800 dm-1 253 2 82169856 dm-2 or + cat /proc/partitions major minor #blocks name 8 0 71652960 sda 8 1 512000 sda1 8 2 71139328 sda2 253 0 8142848 dm-0 253 1 42328064 dm-1 253 2 20664320 dm-2 > But what I see here is that there are partitions and LVM, and > everything is using ext4. That's not useful to me. What I'm after is > the output of pvdisplay, vgdisplay and lvdisplay. That's not very > hard... + pvdisplay --- Physical volume --- PV Name /dev/sda2 VG Name rhel_ibm-ls41-02 PV Size 67.88 GiB / not usable 0 Allocatable yes (but full) PE Size 4.00 MiB Total PE 17376 Free PE 0 Allocated PE 17376 PV UUID Y2xyqY-0Bcf-4NRi-9fmD-fYCm-XD8a-wtvwX6 --- Physical volume --- PV Name /dev/sdb1 VG Name rhel_ibm-ls41-02 PV Size 68.36 GiB / not usable 0 Allocatable yes (but full) PE Size 4.00 MiB Total PE 17501 Free PE 0 Allocated PE 17501 PV UUID RcT6SU-EP1j-Ig3S-pBD0-21p6-C3W1-ersGf9 + echo + vgdisplay --- Volume group --- VG Name rhel_ibm-ls41-02 System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 4 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 2 Act PV 2 VG Size 136.24 GiB PE Size 4.00 MiB Total PE 34877 Alloc PE / Size 34877 / 136.24 GiB Free PE / Size 0 / 0 VG UUID vciDnL-1VvD-Xxx1-caV8-06EH-cZLQ-f9X0vI + echo + lvdisplay --- Logical volume --- LV Path /dev/rhel_ibm-ls41-02/swap LV Name swap VG Name rhel_ibm-ls41-02 LV UUID EgGvG9-XKJ0-sVBq-iQGQ-3jAc-rhbX-ZtCkPK LV Write Access read/write LV Status available # open 2 LV Size 7.88 GiB Current LE 2016 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Path /dev/rhel_ibm-ls41-02/home LV Name home VG Name rhel_ibm-ls41-02 LV UUID RFonzm-OXSb-rVKo-zZRi-YMri-fBC8-GdyRkq LV Write Access read/write LV Status available # open 1 LV Size 78.36 GiB Current LE 20061 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2 --- Logical volume --- LV Path /dev/rhel_ibm-ls41-02/root LV Name root VG Name rhel_ibm-ls41-02 LV UUID AXEjh5-BakC-UwOu-AKjW-7JwD-dsoW-u57ffj LV Write Access read/write LV Status available # open 1 LV Size 50.00 GiB Current LE 12800 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1 or + pvdisplay --- Physical volume --- PV Name /dev/sda2 VG Name rhel_storageqe-13 PV Size 67.84 GiB / not usable 4.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 17367 Free PE 0 Allocated PE 17367 PV UUID qJGFib-CawR-Dlhz-gXew-aqt4-erPL-LLtCeE + echo + vgdisplay --- Volume group --- VG Name rhel_storageqe-13 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 4 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 1 Act PV 1 VG Size 67.84 GiB PE Size 4.00 MiB Total PE 17367 Alloc PE / Size 17367 / 67.84 GiB Free PE / Size 0 / 0 VG UUID 21QGVb-IqM8-DBto-Q1kl-23ql-y28i-ouwb0e + echo + lvdisplay --- Logical volume --- LV Path /dev/rhel_storageqe-13/swap LV Name swap VG Name rhel_storageqe-13 LV UUID jBYJTC-BPds-lfo8-52rJ-MEXR-Chn5-tSChwg LV Write Access read/write LV Status available # open 2 LV Size 7.77 GiB Current LE 1988 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Path /dev/rhel_storageqe-13/home LV Name home VG Name rhel_storageqe-13 LV UUID 3lJRDR-rnNO-7BBf-3Iyy-32ho-Sbnb-qP3Ut1 LV Write Access read/write LV Status available # open 1 LV Size 19.71 GiB Current LE 5045 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2 --- Logical volume --- LV Path /dev/rhel_storageqe-13/root LV Name root VG Name rhel_storageqe-13 LV UUID I2xPqj-Ew0z-NePE-4d7w-nfev-5DMo-fRuFUL LV Write Access read/write LV Status available # open 1 LV Size 40.37 GiB Current LE 10334 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1 I never have had a chance to reproduce this just by running xfstests alone, so it looks like there needs to be some fixture of the loads here. I'll try to see if I can craft out a simpler reproducer... CAI Qian _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-27 5:36 ` Dave Chinner 2013-05-27 6:04 ` CAI Qian 2013-05-28 4:31 ` CAI Qian @ 2013-05-30 3:45 ` CAI Qian 2013-05-30 5:20 ` Dave Chinner 2 siblings, 1 reply; 18+ messages in thread From: CAI Qian @ 2013-05-30 3:45 UTC (permalink / raw) To: Dave Chinner; +Cc: stable, xfs Also reproduced this on 3.9.4 on several different machines. [ 312.686504] BUG: unable to handle kernel paging request at ffff880077434000 [ 312.693518] IP: [<ffffffff81302305>] memmove+0x55/0x1a0 [ 312.698771] PGD 1ddf067 PUD 1de2067 PMD 7748a063 PTE 8000000077434161 [ 312.705284] Oops: 0003 [#1] SMP [ 312.708550] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device e1000e hp_wmi sparse_keymap snd_pcm rfkill ptp iTCO_wdt iTCO_vendor_support pps_core pcspkr mei snd_page_alloc i2c_i801 snd_timer snd lpc_ich soundcore mfd_core microcode xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif nouveau ata_generic video mxm_wmi i2c_algo_bit drm_kms_helper ahci pata_acpi ttm libahci drm libata i2c_core wmi dm_mirror dm_region_hash dm_log dm_mod [ 312.784678] CPU 3 [ 312.786522] Pid: 19267, comm: attr Not tainted 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 312.795590] RIP: 0010:[<ffffffff81302305>] [<ffffffff81302305>] memmove+0x55/0x1a0 [ 312.803273] RSP: 0018:ffff880051d63ad8 EFLAGS: 00010282 [ 312.808596] RAX: ffff880077325688 RBX: ffff880063c00000 RCX: 000000000000d00b [ 312.815739] RDX: ffffffffffef1658 RSI: ffff8800774343e0 RDI: ffff880077433fe8 [ 312.822883] RBP: ffff880051d63b30 R08: 8941c085e1178b44 R09: e8a049a0e0c7c748 [ 312.830020] R10: e0be6090e8c03100 R11: 000001bfa049909d R12: ffff880077325668 [ 312.837160] R13: 00000000000003d8 R14: 0000000000001800 R15: 000000000000007b [ 312.844297] FS: 00007ff3eddd1740(0000) GS:ffff88007d180000(0000) knlGS:0000000000000000 [ 312.852397] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 312.858150] CR2: ffff880077434000 CR3: 0000000063c96000 CR4: 00000000000407e0 [ 312.865285] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 312.872423] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 312.879565] Process attr (pid: 19267, threadinfo ffff880051d62000, task ffff880051a19ac0) [ 312.887750] Stack: [ 312.889764] ffffffffa02651a1 0000000000000000 0000000000000000 ffff880077325688 [ 312.897233] ffff880077325668 0000000000000018 ffff880077325668 0000000000000000 [ 312.904700] ffff8800626e1bd8 ffff880051bccdc0 ffff88004ec26f80 ffff880051d63b68 [ 312.912167] Call Trace: [ 312.914641] [<ffffffffa02651a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [ 312.922240] [<ffffffffa0265467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [ 312.928969] [<ffffffffa0266a2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [ 312.935348] [<ffffffffa0262850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [ 312.942068] [<ffffffffa0270d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [ 312.948613] [<ffffffffa026378c>] xfs_attr_set_int+0x30c/0x420 [xfs] [ 312.954976] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [ 312.960224] [<ffffffffa0263d1f>] xfs_attr_set+0x7f/0x90 [xfs] [ 312.966077] [<ffffffffa0259a12>] xfs_xattr_set+0x42/0x70 [xfs] [ 312.972001] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [ 312.977670] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [ 312.983859] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [ 312.989179] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [ 312.994327] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [ 313.000256] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [ 313.005834] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [ 313.010893] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [ 313.016908] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [ 313.022835] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [ 313.028241] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 313.034252] Code: 40 38 fe 74 41 48 83 ea 20 48 83 ea 20 4c 8b 1e 4c 8b 56 08 4c 8b 4e 10 4c 8b 46 18 48 8d 76 20 4c 89 1f 4c 89 57 08 4c 89 4f 10 <4c> 89 47 18 48 8d 7f 20 73 d4 48 83 c2 20 e9 a8 00 00 00 0f 1f [ 313.054228] RIP [<ffffffff81302305>] memmove+0x55/0x1a0 [ 313.059557] RSP <ffff880051d63ad8> [ 313.063045] CR2: ffff880077434000 [ 313.066362] ---[ end trace 76a02834e4697b82 ]--- [ 313.066368] general protection fault: 0000 [#2] SMP [ 313.066397] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device e1000e hp_wmi sparse_keymap snd_pcm rfkill ptp iTCO_wdt iTCO_vendor_support pps_core pcspkr mei snd_page_alloc i2c_i801 snd_timer snd lpc_ich soundcore mfd_core microcode xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif nouveau ata_generic video mxm_wmi i2c_algo_bit drm_kms_helper ahci pata_acpi ttm libahci drm libata i2c_core wmi dm_mirror dm_region_hash dm_log dm_mod [ 313.066403] CPU 0 [ 313.066404] Pid: 0, comm: swapper/0 Tainted: G D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 313.066409] RIP: 0010:[<ffffffff812fd820>] [<ffffffff812fd820>] timerqueue_add+0x20/0xb0 [ 313.066411] RSP: 0018:ffff88007d003eb0 EFLAGS: 00010002 [ 313.066412] RAX: 53544155415641e5 RBX: ffff88007d00e900 RCX: ffffffffa059ec30 [ 313.066413] RDX: ffffffffa059ec38 RSI: 878b427400407f83 RDI: ffff88007d00e710 [ 313.066414] RBP: ffff88007d003ec0 R08: 00000048a96ae400 R09: 0000000000000000 [ 313.066415] R10: 0000000000000000 R11: 0000000000000005 R12: ffff88007d00e710 [ 313.066416] R13: ffff88007d00e700 R14: ffff88007d00e700 R15: 0000000000000001 [ 313.066417] FS: 0000000000000000(0000) GS:ffff88007d000000(0000) knlGS:0000000000000000 [ 313.066419] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 313.066420] CR2: 0000000000c057a0 CR3: 00000000018fa000 CR4: 00000000000407f0 [ 313.066421] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 313.066422] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 313.066424] Process swapper/0 (pid: 0, threadinfo ffffffff818ee000, task ffffffff81901440) [ 313.066424] Stack: [ 313.066427] ffff88007d00e900 ffff88007d00e900 ffff88007d003ee8 ffffffff81087605 [ 313.066429] ffff88007d00e900 ffff88007d003f58 ffff88007d00e6c0 ffff88007d003f28 [ 313.066431] ffffffff81087b1f ffffffff810ba200 ffff88007d00e6c0 ffff88007d00e6c0 [ 313.066431] Call Trace: [ 313.066433] <IRQ> [ 313.066437] [<ffffffff81087605>] enqueue_hrtimer+0x25/0x80 [ 313.066440] [<ffffffff81087b1f>] __run_hrtimer+0x9f/0x1d0 [ 313.066443] [<ffffffff810ba200>] ? tick_sched_handle.isra.13+0x60/0x60 [ 313.066446] [<ffffffff810882d7>] hrtimer_interrupt+0xe7/0x220 [ 313.066449] [<ffffffff814b8800>] ? cpufreq_p4_cpu_init+0x290/0x290 [ 313.066452] [<ffffffff8161b0e9>] smp_apic_timer_interrupt+0x69/0x9c [ 313.066455] [<ffffffff81619fdd>] apic_timer_interrupt+0x6d/0x80 [ 313.066457] <EOI> [ 313.066459] [<ffffffff814b9181>] ? cpuidle_wrap_enter+0x41/0x80 [ 313.066461] [<ffffffff814b91d0>] cpuidle_enter_tk+0x10/0x20 [ 313.066463] [<ffffffff814b8f32>] cpuidle_idle_call+0xb2/0x1e0 [ 313.066467] [<ffffffff8101c015>] cpu_idle+0xe5/0x140 [ 313.066470] [<ffffffff815edb92>] rest_init+0x72/0x80 [ 313.066475] [<ffffffff81a1bed1>] start_kernel+0x3fd/0x409 [ 313.066477] [<ffffffff81a1b8dd>] ? repair_env_string+0x5c/0x5c [ 313.066481] [<ffffffff81a1b120>] ? early_idt_handlers+0x120/0x120 [ 313.066484] [<ffffffff81a1b5dc>] x86_64_start_reservations+0x2a/0x2c [ 313.066486] [<ffffffff81a1b6d1>] x86_64_start_kernel+0xf3/0x100 [ 313.066505] Code: 31 db eb c6 0f 0b 90 90 90 90 90 55 48 89 e5 41 54 49 89 fc 53 48 3b 36 48 89 f3 75 6c 4c 89 e2 31 c9 eb 1e 0f 1f 80 00 00 00 00 <48> 8b 70 18 48 39 73 18 48 8d 48 08 48 8d 50 10 48 0f 4d d1 48 [ 313.066508] RIP [<ffffffff812fd820>] timerqueue_add+0x20/0xb0 [ 313.066509] RSP <ffff88007d003eb0> [ 313.066511] ---[ end trace 76a02834e4697b83 ]--- [ 313.066515] Kernel panic - not syncing: Fatal exception in interrupt [ 313.066516] general protection fault: 0000 [#3] SMP [ 313.066544] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device e1000e hp_wmi sparse_keymap snd_pcm rfkill ptp iTCO_wdt iTCO_vendor_support pps_core pcspkr mei snd_page_alloc i2c_i801 snd_timer snd lpc_ich soundcore mfd_core microcode xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif nouveau ata_generic video mxm_wmi i2c_algo_bit drm_kms_helper ahci pata_acpi ttm libahci drm libata i2c_core wmi dm_mirror dm_region_hash dm_log dm_mod [ 313.066550] CPU 1 [ 313.066550] Pid: 0, comm: swapper/1 Tainted: G D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 313.066556] RIP: 0010:[<ffffffff8104bf66>] [<ffffffff8104bf66>] leave_mm+0x26/0x70 [ 313.066558] RSP: 0018:ffff880079991e38 EFLAGS: 00010002 [ 313.066559] RAX: 5a5a5a5a5a5a5a5a RBX: ffff88007d09b170 RCX: 0000000000000004 [ 313.066560] RDX: ffff8800773aa1c0 RSI: ffffffff8199a7a0 RDI: 0000000000000001 [ 313.066561] RBP: ffff880079991e38 R08: 0000000000000068 R09: 00000000000003d6 [ 313.066562] R10: 0000000000000000 R11: 00000000000003d8 R12: 0000000000000004 [ 313.066563] R13: 0000000000000020 R14: 0000000000000003 R15: ffffffff814b8800 [ 313.066564] FS: 0000000000000000(0000) GS:ffff88007d080000(0000) knlGS:0000000000000000 [ 313.066565] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 313.066566] CR2: 00007fc2b88c6e00 CR3: 00000000018fa000 CR4: 00000000000407e0 [ 313.066568] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 313.066569] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 313.066570] Process swapper/1 (pid: 0, threadinfo ffff880079990000, task ffff88007996b580) [ 313.066571] Stack: [ 313.066573] ffff880079991e70 ffffffff81358c35 0000000179991e58 ffff88007d09b170 [ 313.066575] ffffffff8199a7a0 00000048a96b3734 0000000000000004 ffff880079991e80 [ 313.066577] ffffffff814b8819 ffff880079991eb8 ffffffff814b9172 0000000000000004 [ 313.066578] Call Trace: [ 313.066582] [<ffffffff81358c35>] intel_idle+0xf5/0x100 [ 313.066584] [<ffffffff814b8819>] cpuidle_enter+0x19/0x20 [ 313.066586] [<ffffffff814b9172>] cpuidle_wrap_enter+0x32/0x80 [ 313.066588] [<ffffffff814b91d0>] cpuidle_enter_tk+0x10/0x20 [ 313.066590] [<ffffffff814b8f32>] cpuidle_idle_call+0xb2/0x1e0 [ 313.066593] [<ffffffff8101c015>] cpu_idle+0xe5/0x140 [ 313.066597] [<ffffffff815f9a9e>] start_secondary+0x24f/0x251 [ 313.066616] Code: 5d c3 0f 1f 00 66 66 66 66 90 55 65 48 8b 14 25 c0 3c 01 00 65 8b 04 25 c8 3c 01 00 83 f8 01 48 89 e5 74 4c 48 8b 82 c0 02 00 00 <0f> a3 38 19 d2 85 d2 75 09 5d c3 0f 1f 80 00 00 00 00 f0 0f b3 [ 313.066619] RIP [<ffffffff8104bf66>] leave_mm+0x26/0x70 [ 313.066620] RSP <ffff880079991e38> [ 313.066622] ---[ end trace 76a02834e4697b84 ]--- [ 313.066625] BUG: unable to handle kernel paging request at ffffebe000000020 [ 313.066631] IP: [<ffffffff8130a9e1>] __list_del_entry+0x1/0xd0 [ 313.066632] PGD 0 [ 313.066634] Oops: 0000 [#4] SMP [ 313.066663] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device e1000e hp_wmi sparse_keymap snd_pcm rfkill ptp iTCO_wdt iTCO_vendor_support pps_core pcspkr mei snd_page_alloc i2c_i801 snd_timer snd lpc_ich soundcore mfd_core microcode xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif nouveau ata_generic video mxm_wmi i2c_algo_bit drm_kms_helper ahci pata_acpi ttm libahci drm libata i2c_core wmi dm_mirror dm_region_hash dm_log dm_mod [ 313.066668] CPU 2 [ 313.066669] Pid: 17, comm: watchdog/2 Tainted: G D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 313.066673] RIP: 0010:[<ffffffff8130a9e1>] [<ffffffff8130a9e1>] __list_del_entry+0x1/0xd0 [ 313.066674] RSP: 0018:ffff880079b21d78 EFLAGS: 00010206 [ 313.066675] RAX: 000001e000000000 RBX: ffffebe000000020 RCX: 000000000000f142 [ 313.066676] RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffffebe000000020 [ 313.066677] RBP: ffff880079b21d90 R08: ffff880079b20000 R09: 0000000000000001 [ 313.066678] R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800773af2c0 [ 313.066679] R13: ffff8800773af2c0 R14: 0000000000000000 R15: ffff8800799b3580 [ 313.066681] FS: 0000000000000000(0000) GS:ffff88007d100000(0000) knlGS:0000000000000000 [ 313.066682] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 313.066683] CR2: ffffebe000000020 CR3: 00000000018fa000 CR4: 00000000000407e0 [ 313.066684] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 313.066685] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 313.066687] Process watchdog/2 (pid: 17, threadinfo ffff880079b20000, task ffff8800799b3580) [ 313.066687] Stack: [ 313.066690] ffff880079b21d90 ffffffff8130aabd 0000000000000000 ffff880079b21db0 [ 313.066692] ffffffff8104acd7 0000000000000000 ffff8800773af2c0 ffff880079b21dd8 [ 313.066694] ffffffff8105de13 0000000000000000 ffff8800773af2c0 ffff880079998000 [ 313.066694] Call Trace: [ 313.066698] [<ffffffff8130aabd>] ? list_del+0xd/0x30 [ 313.066701] [<ffffffff8104acd7>] pgd_free+0x57/0x80 [ 313.066705] [<ffffffff8105de13>] __mmdrop+0x23/0xa0 [ 313.066709] [<ffffffff810915aa>] finish_task_switch+0xba/0xe0 [ 313.066712] [<ffffffff8160eda0>] __schedule+0x3b0/0x7d0 [ 313.066715] [<ffffffff8160f1e9>] schedule+0x29/0x70 [ 313.066719] [<ffffffff8108be63>] smpboot_thread_fn+0xd3/0x1a0 [ 313.066722] [<ffffffff8108bd90>] ? lg_local_lock_cpu+0x40/0x40 [ 313.066724] [<ffffffff81084240>] kthread+0xc0/0xd0 [ 313.066726] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 313.066729] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [ 313.066731] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 313.066750] Code: f5 84 81 be 21 00 00 00 48 c7 c7 41 ea 84 81 31 c0 e8 b4 65 d5 ff e9 73 ff ff ff 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 <48> 8b 17 48 b9 00 01 10 00 00 00 ad de 48 8b 47 08 48 89 e5 48 [ 313.066753] RIP [<ffffffff8130a9e1>] __list_del_entry+0x1/0xd0 [ 313.066754] RSP <ffff880079b21d78> [ 313.066754] CR2: ffffebe000000020 [ 313.066756] ---[ end trace 76a02834e4697b85 ]--- [ 314.133075] Shutting down cpus with NMI [ 314.136923] drm_kms_helper: panic occurred, switching back to text console [ 314.143850] ============================================================================= [ 314.152039] BUG kmalloc-4096 (Tainted: G D ): Padding overwritten. 0xffff880077327c28-0xffff880077327fff [ 314.162310] ----------------------------------------------------------------------------- [ 314.162310] [ 314.171975] INFO: Slab 0xffffea0001dcc800 objects=7 used=7 fp=0x (null) flags=0x10000000004080 [ 314.181380] Pid: 0, comm: swapper/0 Tainted: G B D 3.9.4 #1 [ 314.187656] Call Trace: [ 314.190104] <IRQ> [<ffffffff81181ed2>] slab_err+0xc2/0xf0 [ 314.195717] [<ffffffff810c6f4e>] ? is_module_text_address+0xe/0x20 [ 314.201991] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 [ 314.208269] [<ffffffffa0107bac>] ? drm_crtc_helper_set_config+0x12c/0xb30 [drm_kms_helper] [ 314.216625] [<ffffffff811820e3>] check_slab+0x73/0x100 [ 314.221859] [<ffffffff81188007>] ? offline_memory_block_cb+0x7/0x30 [ 314.228222] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 [ 314.234498] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 [ 314.239995] [<ffffffffa0107bac>] ? drm_crtc_helper_set_config+0x12c/0xb30 [drm_kms_helper] [ 314.248353] [<ffffffff813ca50b>] ? wait_for_xmitr+0x3b/0xa0 [ 314.254021] [<ffffffff813012ea>] ? delay_tsc+0x4a/0x80 [ 314.259254] [<ffffffff81301238>] ? __const_udelay+0x28/0x30 [ 314.264926] [<ffffffff8118474a>] __kmalloc+0x1da/0x220 [ 314.270161] [<ffffffffa0107bac>] drm_crtc_helper_set_config+0x12c/0xb30 [drm_kms_helper] [ 314.278342] [<ffffffff81062e84>] ? wake_up_klogd+0x34/0x50 [ 314.283925] [<ffffffff810630a7>] ? console_unlock+0x207/0x3f0 [ 314.289775] [<ffffffffa00a50f3>] drm_mode_set_config_internal+0x23/0x50 [drm] [ 314.297005] [<ffffffffa01047b7>] drm_fb_helper_restore_fbdev_mode+0x57/0x80 [drm_kms_helper] [ 314.305538] [<ffffffffa0104825>] drm_fb_helper_force_kernel_mode+0x45/0x80 [drm_kms_helper] [ 314.313981] [<ffffffffa010515b>] drm_fb_helper_panic+0x2b/0x30 [drm_kms_helper] [ 314.321382] [<ffffffff81614e5c>] notifier_call_chain+0x4c/0x70 [ 314.327309] [<ffffffff81614eba>] atomic_notifier_call_chain+0x1a/0x20 [ 314.333846] [<ffffffff8160480b>] panic+0xf6/0x1d7 [ 314.338648] [<ffffffff8161206b>] oops_end+0xdb/0xe0 [ 314.343624] [<ffffffff810168db>] die+0x4b/0x70 [ 314.348160] [<ffffffff81611ada>] do_general_protection+0xda/0x160 [ 314.354349] [<ffffffff81611468>] general_protection+0x28/0x30 [ 314.360201] [<ffffffffa059ec30>] ? cache_release_pipefs+0x30/0x30 [sunrpc] [ 314.367177] [<ffffffffa059ec38>] ? sunrpc_cache_pipe_upcall+0x8/0x1e0 [sunrpc] [ 314.374489] [<ffffffff812fd820>] ? timerqueue_add+0x20/0xb0 [ 314.380158] [<ffffffff81087605>] enqueue_hrtimer+0x25/0x80 [ 314.385738] [<ffffffff81087b1f>] __run_hrtimer+0x9f/0x1d0 [ 314.391231] [<ffffffff810ba200>] ? tick_sched_handle.isra.13+0x60/0x60 [ 314.397855] [<ffffffff810882d7>] hrtimer_interrupt+0xe7/0x220 [ 314.403698] [<ffffffff814b8800>] ? cpufreq_p4_cpu_init+0x290/0x290 [ 314.409975] [<ffffffff8161b0e9>] smp_apic_timer_interrupt+0x69/0x9c [ 314.416340] [<ffffffff81619fdd>] apic_timer_interrupt+0x6d/0x80 [ 314.422355] <EOI> [<ffffffff814b9181>] ? cpuidle_wrap_enter+0x41/0x80 [ 314.428997] [<ffffffff814b91d0>] cpuidle_enter_tk+0x10/0x20 [ 314.434667] [<ffffffff814b8f32>] cpuidle_idle_call+0xb2/0x1e0 [ 314.440504] [<ffffffff8101c015>] cpu_idle+0xe5/0x140 [ 314.445567] [<ffffffff815edb92>] rest_init+0x72/0x80 [ 314.450629] [<ffffffff81a1bed1>] start_kernel+0x3fd/0x409 [ 314.456122] [<ffffffff81a1b8dd>] ? repair_env_string+0x5c/0x5c [ 314.462055] [<ffffffff81a1b120>] ? early_idt_handlers+0x120/0x120 [ 314.468246] [<ffffffff81a1b5dc>] x86_64_start_reservations+0x2a/0x2c [ 314.474697] [<ffffffff81a1b6d1>] x86_64_start_kernel+0xf3/0x100 [ 314.480716] Padding ffff8800773278f8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.490117] Padding ffff880077327908: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.499518] Padding ffff880077327918: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.508918] Padding ffff880077327928: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.518312] Padding ffff880077327938: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.527715] Padding ffff880077327948: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.537116] Padding ffff880077327958: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.546511] Padding ffff880077327968: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.555911] Padding ffff880077327978: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.565303] Padding ffff880077327988: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.574700] Padding ffff880077327998: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.584099] Padding ffff8800773279a8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.593491] Padding ffff8800773279b8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.602885] Padding ffff8800773279c8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.612278] Padding ffff8800773279d8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.621680] Padding ffff8800773279e8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.631080] Padding ffff8800773279f8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.640473] Padding ffff880077327a08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.649868] Padding ffff880077327a18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.659263] Padding ffff880077327a28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.668663] Padding ffff880077327a38: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.678055] Padding ffff880077327a48: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.687453] Padding ffff880077327a58: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.696858] Padding ffff880077327a68: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.706252] Padding ffff880077327a78: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.715649] Padding ffff880077327a88: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.725044] Padding ffff880077327a98: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.734442] Padding ffff880077327aa8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.743846] Padding ffff880077327ab8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.753251] Padding ffff880077327ac8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.762646] Padding ffff880077327ad8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.772043] Padding ffff880077327ae8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.781440] Padding ffff880077327af8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.790844] Padding ffff880077327b08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.800250] Padding ffff880077327b18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.809656] Padding ffff880077327b28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.819058] Padding ffff880077327b38: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.828454] Padding ffff880077327b48: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.837858] Padding ffff880077327b58: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.847260] Padding ffff880077327b68: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.856666] Padding ffff880077327b78: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.866059] Padding ffff880077327b88: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.875452] Padding ffff880077327b98: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.884847] Padding ffff880077327ba8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.894248] Padding ffff880077327bb8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.903641] Padding ffff880077327bc8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.913036] Padding ffff880077327bd8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.922435] Padding ffff880077327be8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.931835] Padding ffff880077327bf8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.941229] Padding ffff880077327c08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.950625] Padding ffff880077327c18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 314.960029] Padding ffff880077327c28: 58 44 32 44 00 b8 00 08 00 e8 00 08 01 a0 00 08 XD2D............ [ 314.969430] Padding ffff880077327c38: 00 00 00 00 0c 00 00 8a 01 2e 00 01 00 00 00 10 ................ [ 314.978827] Padding ffff880077327c48: 00 00 00 00 00 00 00 9d 02 2e 2e 00 00 00 00 20 ............... [ 314.988230] Padding ffff880077327c58: 00 00 00 00 0c 02 46 f8 09 67 65 74 73 65 62 6f ......F..getsebo [ 314.997632] Padding ffff880077327c68: 6f 6c 00 00 00 00 00 30 00 00 00 00 0c 04 3e 14 ol.....0......>. [ 315.007034] Padding ffff880077327c78: 05 70 61 72 74 78 00 48 00 00 00 00 0c 00 05 bf .partx.H........ [ 315.016429] Padding ffff880077327c88: 14 62 75 69 6c 64 2d 6c 6f 63 61 6c 65 2d 61 72 .build-locale-ar [ 315.025833] Padding ffff880077327c98: 63 68 69 76 65 00 00 58 00 00 00 00 0c 04 0f 98 chive..X........ [ 315.035237] Padding ffff880077327ca8: 03 6c 69 64 6f 75 00 78 00 00 00 00 0c 04 46 6f .lidou.x......Fo [ 315.044633] Padding ffff880077327cb8: 04 68 61 6c 74 00 00 88 00 00 00 00 0c 00 05 c0 .halt........... [ 315.054031] Padding ffff880077327cc8: 05 7a 64 75 6d 70 00 98 00 00 00 00 0c 04 3e 07 .zdump........>. [ 315.063435] Padding ffff880077327cd8: 04 66 73 63 6b 00 00 a8 ff ff 00 08 00 00 00 b8 .fsck........... [ 315.072838] Padding ffff880077327ce8: 00 00 00 00 0c 00 05 c1 03 7a 69 63 00 00 00 c0 .........zic.... [ 315.082240] Padding ffff880077327cf8: 00 00 00 00 0c 04 0f 97 09 6c 67 72 6f 75 70 6d .........lgroupm [ 315.091642] Padding ffff880077327d08: 6f 64 00 00 00 00 00 d0 ff ff 00 08 00 00 00 e8 od.............. [ 315.101043] Padding ffff880077327d18: 00 00 00 00 0c 00 05 c6 08 6c 64 63 6f 6e 66 69 .........ldconfi [ 315.110435] Padding ffff880077327d28: 67 00 00 00 00 00 00 f0 00 00 00 00 0c 04 3e 08 g.............>. [ 315.119827] Padding ffff880077327d38: 0b 66 73 63 6b 2e 63 72 61 6d 66 73 00 00 01 08 .fsck.cramfs.... [ 315.129218] Padding ffff880077327d48: 00 00 00 00 0c 00 05 c7 03 73 6c 6e 00 00 01 20 .........sln... [ 315.138611] Padding ffff880077327d58: 00 00 00 00 0c 02 46 f7 0a 67 65 74 65 6e 66 6f ......F..getenfo [ 315.148004] Padding ffff880077327d68: 72 63 65 00 00 00 01 30 00 00 00 00 0c 04 3e 09 rce....0......>. [ 315.157397] Padding ffff880077327d78: 0a 66 73 63 6b 2e 6d 69 6e 69 78 00 00 00 01 48 .fsck.minix....H [ 315.166793] Padding ffff880077327d88: 00 00 00 00 0c 00 05 c8 19 67 6c 69 62 63 5f 70 .........glibc_p [ 315.176195] Padding ffff880077327d98: 6f 73 74 5f 75 70 67 72 61 64 65 2e 78 38 36 5f ost_upgrade.x86_ [ 315.185589] Padding ffff880077327da8: 36 34 00 00 00 00 01 60 00 00 00 00 0c 04 0f 99 64.....`........ [ 315.194981] Padding ffff880077327db8: 09 6c 6e 65 77 75 73 65 72 73 64 63 61 3b 01 88 .lnewusersdca;.. [ 315.204374] Padding ffff880077327dc8: ff ff 00 08 63 61 01 a0 00 00 00 00 0c 02 46 f9 ....ca........F. [ 315.213777] Padding ffff880077327dd8: 0c 6d 61 74 63 68 70 61 74 68 63 6f 6e 76 01 a8 .matchpathconv.. [ 315.223179] Padding ffff880077327de8: 00 00 00 00 0c 04 46 70 04 69 6e 69 74 00 01 c0 ......Fp.init... [ 315.232581] Padding ffff880077327df8: 00 00 00 00 0c 00 05 d5 0b 69 63 6f 6e 76 63 6f .........iconvco [ 315.241976] Padding ffff880077327e08: 6e 66 69 67 00 00 01 d0 00 00 00 00 0c 00 05 d5 nfig............ [ 315.251375] Padding ffff880077327e18: 12 69 63 6f 6e 76 63 6f 6e 66 69 67 2e 78 38 36 .iconvconfig.x86 [ 315.260771] Padding ffff880077327e28: 5f 36 34 be 00 00 01 e8 00 00 00 00 0c 04 0f 9a _64............. [ 315.270170] Padding ffff880077327e38: 07 6c 70 61 73 73 77 64 72 73 3b 35 31 61 02 08 .lpasswdrs;51a.. [ 315.279568] Padding ffff880077327e48: ff ff 00 08 00 00 02 20 00 00 00 00 0c 00 05 da ....... ........ [ 315.288971] Padding ffff880077327e58: 05 63 61 70 73 68 02 28 00 00 00 00 0c 04 0f 9b .capsh.(........ [ 315.298369] Padding ffff880077327e68: 08 6c 75 73 65 72 61 64 64 35 31 61 32 38 02 38 .luseradd51a28.8 [ 315.307774] Padding ffff880077327e78: ff ff 00 08 00 00 02 50 00 00 00 00 0c 00 05 db .......P........ [ 315.317179] Padding ffff880077327e88: 06 67 65 74 63 61 70 00 00 00 00 00 00 00 02 58 .getcap........X [ 315.326576] Padding ffff880077327e98: 00 00 00 00 0c 04 0f 9c 08 6c 75 73 65 72 64 65 .........luserde [ 315.335979] Padding ffff880077327ea8: 6c 3b 35 31 61 32 02 70 ff ff 00 08 00 00 02 88 l;51a2.p........ [ 315.345383] Padding ffff880077327eb8: 00 00 00 00 0c 00 05 dc 08 67 65 74 70 63 61 70 .........getpcap [ 315.354787] Padding ffff880077327ec8: 73 00 00 00 00 00 02 90 00 00 00 00 0c 04 0f 9d s............... [ 315.364190] Padding ffff880077327ed8: 08 6c 75 73 65 72 6d 6f 64 3b 35 31 61 32 02 a8 .lusermod;51a2.. [ 315.373588] Padding ffff880077327ee8: ff ff 00 08 00 00 02 c0 00 00 00 00 0c 00 05 dd ................ [ 315.382993] Padding ffff880077327ef8: 06 73 65 74 63 61 70 00 00 00 00 00 00 00 02 c8 .setcap......... [ 315.392390] Padding ffff880077327f08: 00 00 00 00 0c 03 0e fe 09 76 70 64 64 65 63 6f .........vpddeco [ 315.401794] Padding ffff880077327f18: 64 65 6e 6c 69 73 02 e0 00 00 00 00 0c 04 48 ac denlis........H. [ 315.411199] Padding ffff880077327f28: 05 63 72 6f 6e 64 02 f8 00 00 00 00 0c 00 a8 c1 .crond.......... [ 315.420603] Padding ffff880077327f38: 12 73 65 66 63 6f 6e 74 65 78 74 5f 63 6f 6d 70 .sefcontext_comp [ 315.430006] Padding ffff880077327f48: 69 6c 65 00 00 00 03 08 00 00 00 00 0c 04 2d a7 ile...........-. [ 315.439405] Padding ffff880077327f58: 06 63 68 72 6f 6f 74 6f 64 3b 35 31 61 32 03 28 .chrootod;51a2.( [ 315.448807] Padding ffff880077327f68: ff ff 00 08 63 61 03 40 00 00 00 00 0c 00 a8 c2 ....ca.@........ [ 315.458205] Padding ffff880077327f78: 0c 69 6e 73 74 61 6c 6c 2d 69 6e 66 6f 00 03 48 .install-info..H [ 315.467609] Padding ffff880077327f88: 00 00 00 00 0c 04 2d c9 08 66 61 69 6c 6c 6f 63 ......-..failloc [ 315.477012] Padding ffff880077327f98: 6b 31 61 32 38 64 03 60 ff ff 00 08 00 00 03 78 k1a28d.`.......x [ 315.486408] Padding ffff880077327fa8: 00 00 00 00 0c 00 a8 cb 09 63 68 6b 63 6f 6e 66 .........chkconf [ 315.495804] Padding ffff880077327fb8: 69 67 00 00 00 00 03 80 00 00 00 00 0c 04 2d cb ig............-. [ 315.505209] Padding ffff880077327fc8: 11 70 61 6d 5f 63 6f 6e 73 6f 6c 65 5f 61 70 70 .pam_console_app [ 315.514605] Padding ffff880077327fd8: 6c 79 38 64 63 61 03 98 00 00 00 00 0c 00 a8 cc ly8dca.......... [ 315.524008] Padding ffff880077327fe8: 0c 61 6c 74 65 72 6e 61 74 69 76 65 73 00 03 b8 .alternatives... [ 315.533408] Padding ffff880077327ff8: 00 00 00 00 0c 02 46 fa ......F. [ 315.542107] FIX kmalloc-4096: Restoring 0xffff8800773278f8-0xffff880077327fff=0x5a System information is below: + cat /proc/mounts rootfs / rootfs rw 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=1984960k,nr_inodes=496240,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs rw,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 /dev/mapper/rhel_hp--xw9400--02-root / xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 tmpfs /tmp tmpfs rw,seclabel 0 0 /dev/mapper/rhel_hp--xw9400--02-home /home xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 /dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 + echo + lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 12 On-line CPU(s) list: 0-11 Thread(s) per core: 1 Core(s) per socket: 6 Socket(s): 2 NUMA node(s): 2 Vendor ID: AuthenticAMD CPU family: 16 Model: 8 Model name: Six-Core AMD Opteron(tm) Processor 2427 Stepping: 0 CPU MHz: 800.000 BogoMIPS: 4400.02 Virtualization: AMD-V L1d cache: 64K L1i cache: 64K L2 cache: 512K L3 cache: 6144K NUMA node0 CPU(s): 0,2,4,6,8,10 NUMA node1 CPU(s): 1,3,5,7,9,11 + echo + cat /proc/partitions major minor #blocks name 11 0 1536 sr0 8 0 39082680 sda 8 1 512000 sda1 8 2 38568960 sda2 253 0 4063232 dm-0 8 16 35566480 sdb 8 17 35564544 sdb1 253 1 47079424 dm-1 253 2 22986752 dm-2 + echo + pvdisplay --- Physical volume --- PV Name /dev/sda2 VG Name rhel_hp-xw9400-02 PV Size 36.78 GiB / not usable 0 Allocatable yes (but full) PE Size 4.00 MiB Total PE 9416 Free PE 0 Allocated PE 9416 PV UUID 8MvTYN-OaUs-a7K4-yF0W-k1G6-eItC-ndZDel --- Physical volume --- PV Name /dev/sdb1 VG Name rhel_hp-xw9400-02 PV Size 33.92 GiB / not usable 3.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 8682 Free PE 0 Allocated PE 8682 PV UUID 43zey8-rZMQ-VCO3-PnTM-fb8v-3IJF-LA9FvI + echo + vgdisplay --- Volume group --- VG Name rhel_hp-xw9400-02 System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 4 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 2 Act PV 2 VG Size 70.70 GiB PE Size 4.00 MiB Total PE 18098 Alloc PE / Size 18098 / 70.70 GiB Free PE / Size 0 / 0 VG UUID 923eYU-qAJR-kRbj-MYwR-zBeh-2lP8-ZyhOfF + echo + lvdisplay --- Logical volume --- LV Path /dev/rhel_hp-xw9400-02/swap LV Name swap VG Name rhel_hp-xw9400-02 LV UUID FvWm56-fypL-T2du-CxVW-riyL-tO06-8dLBUH LV Write Access read/write LV Status available # open 2 LV Size 3.88 GiB Current LE 992 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Path /dev/rhel_hp-xw9400-02/home LV Name home VG Name rhel_hp-xw9400-02 LV UUID 8VXYot-OHHf-WaXA-RlFj-I9b5-7qhc-zGZ5Qx LV Write Access read/write LV Status available # open 1 LV Size 21.92 GiB Current LE 5612 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2 --- Logical volume --- LV Path /dev/rhel_hp-xw9400-02/root LV Name root VG Name rhel_hp-xw9400-02 LV UUID R5qalV-W0Dw-3C0S-JfUE-ubnB-plP9-j6Gf8D LV Write Access read/write LV Status available # open 1 LV Size 44.90 GiB Current LE 11494 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1 CAI Qian _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-30 3:45 ` CAI Qian @ 2013-05-30 5:20 ` Dave Chinner 2013-05-30 5:40 ` CAI Qian 0 siblings, 1 reply; 18+ messages in thread From: Dave Chinner @ 2013-05-30 5:20 UTC (permalink / raw) To: CAI Qian; +Cc: stable, xfs On Wed, May 29, 2013 at 11:45:28PM -0400, CAI Qian wrote: > Also reproduced this on 3.9.4 on several different machines. How? what workload are you running that causes it to occur? <snip stack trace showing page overrun in xfs_attr_leaf_compact/xfs_attr_leaf_moveents> > System information is below: > > + cat /proc/mounts .... > /dev/mapper/rhel_hp--xw9400--02-root / xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 > /dev/mapper/rhel_hp--xw9400--02-home /home xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 > /dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 Which of these filesystems did it happen on? Or some other filesystem not mounted when you took this snapshot? And can you post the xfs_info output of the filesystems under test, too? Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.3: Oops running xfstests 2013-05-30 5:20 ` Dave Chinner @ 2013-05-30 5:40 ` CAI Qian 2013-05-31 3:03 ` 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) CAI Qian 0 siblings, 1 reply; 18+ messages in thread From: CAI Qian @ 2013-05-30 5:40 UTC (permalink / raw) To: Dave Chinner; +Cc: stable, xfs ----- Original Message ----- > From: "Dave Chinner" <david@fromorbit.com> > To: "CAI Qian" <caiqian@redhat.com> > Cc: xfs@oss.sgi.com, stable@vger.kernel.org > Sent: Thursday, May 30, 2013 1:20:49 PM > Subject: Re: 3.9.3: Oops running xfstests > > On Wed, May 29, 2013 at 11:45:28PM -0400, CAI Qian wrote: > > Also reproduced this on 3.9.4 on several different machines. > > How? what workload are you running that causes it to occur? I have been running the following workloads, trinity, ltp (including oom tests and some filesystem tests), some hugetlbfs tests, selinux test suite, tracepoint tests, connectathon tests and then xfstests. I am still trying to narrow down the minimal workload that could reproduce this. > > <snip stack trace showing page overrun in > xfs_attr_leaf_compact/xfs_attr_leaf_moveents> > > > System information is below: > > > > + cat /proc/mounts > .... > > /dev/mapper/rhel_hp--xw9400--02-root / xfs > > rw,seclabel,relatime,attr2,inode64,noquota 0 0 > > /dev/mapper/rhel_hp--xw9400--02-home /home xfs > > rw,seclabel,relatime,attr2,inode64,noquota 0 0 > > /dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 > > Which of these filesystems did it happen on? Or some other > filesystem not mounted when you took this snapshot? + export TEST_DEV=/dev/mapper/rhel_hp--bl495cg5--02-swap + TEST_DEV=/dev/mapper/rhel_hp--bl495cg5--02-swap + export TEST_DIR=/mnt/testarea/test + TEST_DIR=/mnt/testarea/test + export SCRATCH_DEV=/dev/mapper/rhel_hp--bl495cg5--02-home + SCRATCH_DEV=/dev/mapper/rhel_hp--bl495cg5--02-home + export SCRATCH_MNT=/mnt/testarea/scratch + SCRATCH_MNT=/mnt/testarea/scratch + mkdir -p /mnt/testarea/test home = /dev/mapper/rhel_hp--bl495cg5--02-home + mkdir -p /mnt/testarea/scratch + mkfs.xfs -f /dev/mapper/rhel_hp--bl495cg5--02-swap meta-data=/dev/mapper/rhel_hp--bl495cg5--02-swap isize=256 agcount=4, agsize=510976 blks = sectsz=512 attr=2, projid32bit=0 data = bsize=4096 blocks=2043904, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 + mkfs.xfs -f /dev/mapper/rhel_hp--bl495cg5--02-home meta-data=/dev/mapper/rhel_hp--bl495cg5--02-home isize=256 agcount=4, agsize=1103872 blks = sectsz=512 attr=2, projid32bit=0 data = bsize=4096 blocks=4415488, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 > > And can you post the xfs_info output of the filesystems under test, > too? xfs_info: /dev/mapper/rhel_hp--bl495cg5--02-swap is not a mounted XFS filesystem + echo + xfs_info /dev/mapper/rhel_hp--bl495cg5--02-home meta-data=/dev/mapper/rhel_hp--bl495cg5--02-home isize=256 agcount=4, agsize=1103872 blks = sectsz=512 attr=2 data = bsize=4096 blocks=4415488, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 CAI Qian > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com > _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-05-30 5:40 ` CAI Qian @ 2013-05-31 3:03 ` CAI Qian 2013-05-31 5:35 ` Eric Sandeen 2013-05-31 6:04 ` Dave Chinner 0 siblings, 2 replies; 18+ messages in thread From: CAI Qian @ 2013-05-31 3:03 UTC (permalink / raw) To: Dave Chinner; +Cc: linux-mm, LKML, stable, xfs OK, so the minimal workload to trigger this I found so far was to run trinity, ltp and then xfstests. I have been able to easily reproduced on 3 servers so far, and I'll post full logs here for LKML and linux-mm as this may unrelated to XFS only. As far as I can tell from the previous testing results, this has never been reproduced back in 3.9 GA time. This seems also been reproduced on 3.10-rc3 mostly on s390x so far. CAI Qian 1) [10948.104344] XFS (dm-2): Mounting Filesystem [10948.257335] XFS (dm-2): Ending clean mount [10949.753951] XFS (dm-0): Mounting Filesystem [10950.012824] XFS (dm-0): Ending clean mount [10951.115722] BUG: unable to handle kernel paging request at ffff88003a9ca478 [10951.122727] IP: [<ffffffff810f7f47>] __call_rcu.constprop.47+0x77/0x230 [10951.129372] PGD 1ddf067 PUD 1de0067 PMD 3da02063 PTE 889b2731bac1a751 [10951.135887] Oops: 000b [#1] SMP [10951.139157] Modules linked in: binfmt_misc(F) fuse(F) snd_seq_dummy(F) rfcomm(F) hidp(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) cmtp(F) kernelcapi(F) tun(F) bnep(F) ipt_ULOG(F) scsi_transport_iscsi(F) nfnetlink(F) rds(F) bluetooth(F) af_key(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) atm(F) nfc(F) af_802154(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) snd_hda_codec_hdmi(F) snd_hda_codec_realtek(F) snd_hda_intel(F) snd_hda_codec(F) snd_hwdep(F) snd_seq(F) snd_seq_device(F) snd_pcm(F) hp_wmi(F) sparse_keymap(F) e1000e(F) rfkill(F) iTCO_wdt(F) snd_page_alloc(F) snd_timer(F) snd(F) iTCO_ vendor_support(F) soundcore(F) mei(F) ptp(F) i2c_i801(F) pps_core(F) lpc_ich(F) pcspkr(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sr_mod(F) sd_mod(F) cdrom(F) crc_t10dif(F) nouveau(F) video(F) mxm_wmi(F) i2c_algo_bit(F) drm_kms_helper(F) ata_generic(F) ttm(F) ahci(F) libahci(F) pata_acpi(F) drm(F) libata(F) i2c_core(F) wmi(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10951.260278] CPU 1 [10951.262122] Pid: 0, comm: swapper/1 Tainted: GF 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [10951.272238] RIP: 0010:[<ffffffff810f7f47>] [<ffffffff810f7f47>] __call_rcu.constprop.47+0x77/0x230 [10951.281307] RSP: 0018:ffff88007d083e00 EFLAGS: 00010046 [10951.286620] RAX: ffff88003a9ca478 RBX: ffff88007d08ec80 RCX: 0000000000000000 [10951.293763] RDX: ffffffff81947c80 RSI: ffffffff8108a350 RDI: ffff88005dbcd890 [10951.300909] RBP: ffff88007d083e28 R08: 0000000000000000 R09: 0000000000000000 [10951.308051] R10: ffff88007cc2c480 R11: ffffea000106ab00 R12: ffff88005dbcd890 [10951.315187] R13: 0000000000000246 R14: ffff880041aad520 R15: ffff88007d08ec80 [10951.322324] FS: 0000000000000000(0000) GS:ffff88007d080000(0000) knlGS:0000000000000000 [10951.330422] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10951.336176] CR2: ffff88003a9ca478 CR3: 00000000018fa000 CR4: 00000000000407e0 [10951.343319] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10951.350463] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10951.357604] Process swapper/1 (pid: 0, threadinfo ffff880079990000, task ffff88007996b580) [10951.365872] Stack: [10951.367885] ffff88007996b580 000000010012000b ffff880079955040 ffff880079955040 [10951.375347] ffff8800799559f8 ffff88007d083e38 ffffffff810f8157 ffff88007d083e48 [10951.382812] ffffffff8108a4af ffff88007d083e60 ffffffff8108a958 ffff880079955040 [10951.390276] Call Trace: [10951.392723] <IRQ> [10951.394658] [<ffffffff810f8157>] call_rcu_sched+0x17/0x20 [10951.400353] [<ffffffff8108a4af>] __put_cred+0x3f/0x50 [10951.405499] [<ffffffff8108a958>] exit_creds+0x58/0x60 [10951.410647] [<ffffffff8105e0ca>] __put_task_struct+0x4a/0x140 [10951.416485] [<ffffffff8106519c>] delayed_put_task_struct+0x3c/0x80 [10951.422759] [<ffffffff810f833d>] rcu_process_callbacks+0x1dd/0x550 [10951.429035] [<ffffffff8106991f>] __do_softirq+0xef/0x230 [10951.434442] [<ffffffff814b8800>] ? cpufreq_p4_cpu_init+0x290/0x290 [10951.440718] [<ffffffff81069be5>] irq_exit+0xa5/0xb0 [10951.445691] [<ffffffff8161b0ee>] smp_apic_timer_interrupt+0x6e/0x9c [10951.452055] [<ffffffff81619fdd>] apic_timer_interrupt+0x6d/0x80 [10951.458065] <EOI> [10951.459995] [<ffffffff814b9181>] ? cpuidle_wrap_enter+0x41/0x80 [10951.466218] [<ffffffff814b91d0>] cpuidle_enter_tk+0x10/0x20 [10951.471885] [<ffffffff814b8f32>] cpuidle_idle_call+0xb2/0x1e0 [10951.477730] [<ffffffff8101c015>] cpu_idle+0xe5/0x140 [10951.482790] [<ffffffff815f9a9e>] start_secondary+0x24f/0x251 [10951.488542] Code: c8 db 00 00 48 83 7b 48 00 0f 84 f1 00 00 00 48 8b 43 78 48 83 c0 01 84 c9 48 89 43 78 74 05 48 83 43 70 01 0f ae f0 48 8b 43 48 <4c> 89 20 4c 89 63 48 48 c7 c1 20 eb 00 00 65 48 03 0c 25 c8 db [10951.508531] RIP [<ffffffff810f7f47>] __call_rcu.constprop.47+0x77/0x230 [10951.515258] RSP <ffff88007d083e00> [10951.518748] CR2: ffff88003a9ca478 [10951.527448] ---[ end trace 1dba1edcdfef3999 ]--- [10951.527453] BUG: unable to handle kernel paging request at ffff88003db52000 [10951.527459] IP: [<ffffffff81302301>] memmove+0x51/0x1a0 [10951.527462] PGD 1ddf067 PUD 1de0067 PMD 626ab063 PTE 800000003db52161 [10951.527464] Oops: 0003 [#2] SMP [10951.527487] Modules linked in: binfmt_misc(F) fuse(F) snd_seq_dummy(F) rfcomm(F) hidp(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) cmtp(F) kernelcapi(F) tun(F) bnep(F) ipt_ULOG(F) scsi_transport_iscsi(F) nfnetlink(F) rds(F) bluetooth(F) af_key(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) atm(F) nfc(F) af_802154(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) snd_hda_codec_hdmi(F) snd_hda_codec_realtek(F) snd_hda_intel(F) snd_hda_codec(F) snd_hwdep(F) snd_seq(F) snd_seq_device(F) snd_pcm(F) hp_wmi(F) sparse_keymap(F) e1000e(F) rfkill(F) iTCO_wdt(F) snd_page_alloc(F) snd_timer(F) snd(F) iTCO_ vendor_support(F) soundcore(F) mei(F) ptp(F) i2c_i801(F) pps_core(F) lpc_ich(F) pcspkr(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sr_mod(F) sd_mod(F) cdrom(F) crc_t10dif(F) nouveau(F) video(F) mxm_wmi(F) i2c_algo_bit(F) drm_kms_helper(F) ata_generic(F) ttm(F) ahci(F) libahci(F) pata_acpi(F) drm(F) libata(F) i2c_core(F) wmi(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10951.527508] CPU 0 [10951.527509] Pid: 8341, comm: attr Tainted: GF D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [10951.527513] RIP: 0010:[<ffffffff81302301>] [<ffffffff81302301>] memmove+0x51/0x1a0 [10951.527514] RSP: 0018:ffff88003106fad8 EFLAGS: 00010286 [10951.527515] RAX: ffff88003d95a2b0 RBX: ffff880042652000 RCX: 000000000000d00b [10951.527516] RDX: ffffffffffe08278 RSI: ffff88003db523e8 RDI: ffff88003db51ff0 [10951.527517] RBP: ffff88003106fb30 R08: 15e8a066d060c7c7 R09: 481974030000012c [10951.527518] R10: b883660a74c08548 R11: a066db00dd048b48 R12: ffff88003d95a290 [10951.527519] R13: 00000000000003d8 R14: 0000000000001800 R15: 000000000000007b [10951.527521] FS: 00007feda92ea740(0000) GS:ffff88007d000000(0000) knlGS:0000000000000000 [10951.527522] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10951.527523] CR2: ffff88003db52000 CR3: 000000005ab9f000 CR4: 00000000000407f0 [10951.527524] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10951.527525] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10951.527527] Process attr (pid: 8341, threadinfo ffff88003106e000, task ffff88003a9c8000) [10951.527527] Stack: [10951.527530] ffffffffa028c1a1 0000000000000000 0000000000000000 ffff88003d95a2b0 [10951.527532] ffff88003d95a290 0000000000000018 ffff88003d95a290 0000000000000000 [10951.527534] ffff880077bd4a40 ffff880063fe63c0 ffff880042c9fb60 ffff88003106fb68 [10951.527535] Call Trace: [10951.527563] [<ffffffffa028c1a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [10951.527582] [<ffffffffa028c467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [10951.527602] [<ffffffffa028da2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [10951.527619] [<ffffffffa0289850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [10951.527638] [<ffffffffa0297d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [10951.527656] [<ffffffffa028a78c>] xfs_attr_set_int+0x30c/0x420 [xfs] [10951.527660] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [10951.527678] [<ffffffffa028ad1f>] xfs_attr_set+0x7f/0x90 [xfs] [10951.527696] [<ffffffffa0280a12>] xfs_xattr_set+0x42/0x70 [xfs] [10951.527699] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [10951.527702] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [10951.527705] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [10951.527707] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [10951.527711] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [10951.527713] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [10951.527715] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [10951.527717] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [10951.527720] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [10951.527723] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [10951.527727] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [10951.527746] Code: 00 00 72 05 40 38 fe 74 41 48 83 ea 20 48 83 ea 20 4c 8b 1e 4c 8b 56 08 4c 8b 4e 10 4c 8b 46 18 48 8d 76 20 4c 89 1f 4c 89 57 08 <4c> 89 4f 10 4c 89 47 18 48 8d 7f 20 73 d4 48 83 c2 20 e9 a8 00 [10951.527749] RIP [<ffffffff81302301>] memmove+0x51/0x1a0 [10951.527750] RSP <ffff88003106fad8> [10951.527750] CR2: ffff88003db52000 [10951.527752] ---[ end trace 1dba1edcdfef399a ]--- [10951.527814] BUG: unable to handle kernel paging request at ffff88003a9d2018 [10951.527819] IP: [<ffffffff81095092>] try_to_wake_up+0x52/0x2d0 [10951.527824] PGD 1ddf067 PUD 1de0067 PMD 3da02063 PTE 2d00000000000000 [10951.527826] Oops: 0000 [#3] SMP [10951.527847] Modules linked in: binfmt_misc(F) fuse(F) snd_seq_dummy(F) rfcomm(F) hidp(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) cmtp(F) kernelcapi(F) tun(F) bnep(F) ipt_ULOG(F) scsi_transport_iscsi(F) nfnetlink(F) rds(F) bluetooth(F) af_key(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) atm(F) nfc(F) af_802154(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) snd_hda_codec_hdmi(F) snd_hda_codec_realtek(F) snd_hda_intel(F) snd_hda_codec(F) snd_hwdep(F) snd_seq(F) snd_seq_device(F) snd_pcm(F) hp_wmi(F) sparse_keymap(F) e1000e(F) rfkill(F) iTCO_wdt(F) snd_page_alloc(F) snd_timer(F) snd(F) iTCO_ vendor_support(F) soundcore(F) mei(F) ptp(F) i2c_i801(F) pps_core(F) lpc_ich(F) pcspkr(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sr_mod(F) sd_mod(F) cdrom(F) crc_t10dif(F) nouveau(F) video(F) mxm_wmi(F) i2c_algo_bit(F) drm_kms_helper(F) ata_generic(F) ttm(F) ahci(F) libahci(F) pata_acpi(F) drm(F) libata(F) i2c_core(F) wmi(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10951.527868] CPU 0 [10951.527868] Pid: 8341, comm: attr Tainted: GF D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [10951.527871] RIP: 0010:[<ffffffff81095092>] [<ffffffff81095092>] try_to_wake_up+0x52/0x2d0 [10951.527873] RSP: 0018:ffff88007d003d98 EFLAGS: 00010046 [10951.527874] RAX: ffff88003a9d2000 RBX: ffff8800647f8000 RCX: 0000000000000000 [10951.527875] RDX: 000000003e463e46 RSI: 0000000000000003 RDI: ffff8800647f87ec [10951.527876] RBP: ffff88007d003dd8 R08: ffff88007d013d10 R09: ffffffff81c75303 [10951.527877] R10: 0000000000000000 R11: 000000000000000c R12: ffff8800647f87ec [10951.527878] R13: 0000000000000000 R14: 0000000000000046 R15: 0000000000014400 [10951.527879] FS: 00007feda92ea740(0000) GS:ffff88007d000000(0000) knlGS:0000000000000000 [10951.527880] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10951.527881] CR2: ffff88003a9d2018 CR3: 000000005ab9f000 CR4: 00000000000407f0 [10951.527882] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10951.527883] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10951.527884] Process attr (pid: 8341, threadinfo ffff88003106e000, task ffff88003a9c8000) [10951.527885] Stack: [10951.527887] 0000008000000001 ffff88007d003db0 0000000000000003 ffff8800647f8000 [10951.527889] ffff88007d017e00 ffff88007d010f08 0000000000000000 ffff88007cc428a0 [10951.527891] ffff88007d003df0 ffffffff81095333 ffff88007d013d00 ffff88007d003e00 [10951.527892] Call Trace: [10951.527893] <IRQ> [10951.527896] [<ffffffff81095333>] wake_up_process+0x23/0x40 [10951.527899] [<ffffffff8107c064>] wake_up_worker+0x24/0x30 [10951.527901] [<ffffffff8107c533>] insert_work+0x43/0x50 [10951.527904] [<ffffffff8107c6e8>] __queue_work+0x118/0x270 [10951.527906] [<ffffffff8107c840>] ? __queue_work+0x270/0x270 [10951.527909] [<ffffffff8107c858>] delayed_work_timer_fn+0x18/0x20 [10951.527911] [<ffffffff8106f106>] call_timer_fn+0x36/0x110 [10951.527913] [<ffffffff8107c840>] ? __queue_work+0x270/0x270 [10951.527916] [<ffffffff81071431>] run_timer_softirq+0x151/0x290 [10951.527920] [<ffffffff8106991f>] __do_softirq+0xef/0x230 [10951.527923] [<ffffffff81069be5>] irq_exit+0xa5/0xb0 [10951.527925] [<ffffffff8161b0ee>] smp_apic_timer_interrupt+0x6e/0x9c [10951.527928] [<ffffffff81619fdd>] apic_timer_interrupt+0x6d/0x80 [10951.527929] <EOI> [10951.527932] [<ffffffff810c8c0f>] ? acct_collect+0x16f/0x1b0 [10951.527934] [<ffffffff81066e01>] do_exit+0x7e1/0xa20 [10951.527937] [<ffffffff81604989>] ? printk+0x67/0x69 [10951.527941] [<ffffffff81612034>] oops_end+0xa4/0xe0 [10951.527943] [<ffffffff81604212>] no_context+0x263/0x270 [10951.527945] [<ffffffff81604292>] __bad_area_nosemaphore+0x73/0x1cc [10951.527947] [<ffffffff816043fe>] bad_area_nosemaphore+0x13/0x15 [10951.527949] [<ffffffff81614c0e>] __do_page_fault+0x31e/0x510 [10951.527967] [<ffffffffa0284497>] ? kmem_alloc+0x67/0xf0 [xfs] [10951.527971] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [10951.527974] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [10951.527976] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [10951.527979] [<ffffffff81614e0e>] do_page_fault+0xe/0x10 [10951.527982] [<ffffffff81611498>] page_fault+0x28/0x30 [10951.527985] [<ffffffff81302301>] ? memmove+0x51/0x1a0 [10951.528003] [<ffffffffa028c1a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [10951.528020] [<ffffffffa028c467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [10951.528038] [<ffffffffa028da2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [10951.528055] [<ffffffffa0289850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [10951.528073] [<ffffffffa0297d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [10951.528090] [<ffffffffa028a78c>] xfs_attr_set_int+0x30c/0x420 [xfs] [10951.528093] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [10951.528111] [<ffffffffa028ad1f>] xfs_attr_set+0x7f/0x90 [xfs] [10951.528128] [<ffffffffa0280a12>] xfs_xattr_set+0x42/0x70 [xfs] [10951.528131] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [10951.528134] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [10951.528137] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [10951.528140] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [10951.528142] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [10951.528144] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [10951.528146] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [10951.528148] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [10951.528150] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [10951.528154] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [10951.528157] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [10951.528175] Code: 31 ff 4c 89 e7 e8 3f bd 57 00 8b 75 d0 49 89 c6 48 8b 03 85 c6 0f 84 9f 01 00 00 8b 4b 2c 48 8b 43 08 49 c7 c7 00 44 01 00 85 c9 <44> 8b 48 18 74 7a 49 c7 c7 00 44 01 00 44 89 c8 4c 89 fa 48 89 [10951.528178] RIP [<ffffffff81095092>] try_to_wake_up+0x52/0x2d0 [10951.528179] RSP <ffff88007d003d98> [10951.528180] CR2: ffff88003a9d2018 [10951.528181] ---[ end trace 1dba1edcdfef399b ]--- [10951.528182] Kernel panic - not syncing: Fatal exception in interrupt [10952.594747] Shutting down cpus with NMI [10952.598598] drm_kms_helper: panic occurred, switching back to text console 2) [20914.575680] XFS (dm-2): Mounting Filesystem [20914.653966] XFS (dm-2): Ending clean mount [20915.812263] XFS (dm-0): Mounting Filesystem [20916.628209] XFS (dm-0): Ending clean mount [20917.997069] divide error: 0000 [#1] SMP [20917.998029] Modules linked in: binfmt_misc(F) cmtp(F) kernelcapi(F) tun(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) bnep(F) hidp(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) rds(F) af_key(F) bluetooth(F) rfkill(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) nfc(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) bnx2(F) kvm_amd(F) amd64_edac_mod(F) edac_mce_amd(F) kvm(F) pcspkr(F) serio_raw(F) shpchp(F) i2c_piix4(F) edac_core(F) k8temp(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) radeon(F) i2c_algo_bit(F) drm_kms_helper(F) mptsas(F) ttm(F) scsi_transport_sas(F) drm(F ) mptscsih(F) i2c_core(F) mptbase(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [20917.998029] CPU 0 [20917.998029] Pid: 10, comm: rcu_sched Tainted: GF 3.9.4 #1 IBM BladeCenter LS41 -[7972AC1]-/Server Blade [20917.998029] RIP: 0010:[<ffffffff8109ed2b>] [<ffffffff8109ed2b>] update_sd_lb_stats+0x2bb/0x5b0 [20917.998029] RSP: 0018:ffff88013a35fa60 EFLAGS: 00010016 [20917.998029] RAX: 0000000000044800 RBX: 0000000000000000 RCX: ffff880439f5ba18 [20917.998029] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000008 [20917.998029] RBP: ffff88013a35fb28 R08: 0000000000000112 R09: 0000000000000000 [20917.998029] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000008 [20917.998029] R13: 0000000000000000 R14: 0000000000000001 R15: ffff88013a35fc50 [20917.998029] FS: 00007fb8dd06e740(0000) GS:ffff88013bc00000(0000) knlGS:0000000000000000 [20917.998029] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [20917.998029] CR2: 0000000001890840 CR3: 00000002205ce000 CR4: 00000000000007f0 [20917.998029] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [20917.998029] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [20917.998029] Process rcu_sched (pid: 10, threadinfo ffff88013a35e000, task ffff88013a365040) [20917.998029] Stack: [20917.998029] 0100000000000000 ffff88013a35fce4 ffff88013a35fb38 00000000ffffffff [20917.998029] ffff880139c21fb0 0000000000014400 0000000000014400 0000000000000000 [20917.998029] 0000000000000001 0000000000000112 ffff880139c21fc8 0000000000000000 [20917.998029] Call Trace: [20917.998029] [<ffffffff8109f056>] find_busiest_group+0x36/0x4b0 [20917.998029] [<ffffffff8109f5d8>] load_balance+0x108/0x7d0 [20917.998029] [<ffffffff8108fa7c>] ? update_rq_clock.part.67+0x1c/0x170 [20917.998029] [<ffffffff810a0142>] idle_balance+0x182/0x2f0 [20917.998029] [<ffffffff8160f1ac>] __schedule+0x7bc/0x7d0 [20917.998029] [<ffffffff8160f1e9>] schedule+0x29/0x70 [20917.998029] [<ffffffff8160d8e2>] schedule_timeout+0x172/0x2c0 [20917.998029] [<ffffffff810f709a>] ? rcu_report_qs_rnp+0x7a/0x130 [20917.998029] [<ffffffff81070210>] ? __internal_add_timer+0x130/0x130 [20917.998029] [<ffffffff810f8ce3>] rcu_gp_kthread+0x313/0x560 [20917.998029] [<ffffffff810850e0>] ? wake_up_bit+0x30/0x30 [20917.998029] [<ffffffff810f89d0>] ? rcu_gp_fqs+0x80/0x80 [20917.998029] [<ffffffff81084240>] kthread+0xc0/0xd0 [20917.998029] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [20917.998029] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [20917.998029] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [20917.998029] Code: b5 50 ff ff ff 49 8b 3f e8 b3 fb ff ff 0f 1f 00 48 8b 85 58 ff ff ff 48 8b 48 10 4c 8b 45 a0 8b 71 04 31 d2 4c 89 c0 48 c1 e0 0a <48> f7 f6 48 8b 75 a8 48 85 f6 48 89 c7 49 89 c1 48 89 45 98 0f [20917.998029] RIP [<ffffffff8109ed2b>] update_sd_lb_stats+0x2bb/0x5b0 [20917.998029] RSP <ffff88013a35fa60> [20917.998029] ---[ end trace 9d1c03412e3d1bf5 ]--- [20918.017044] divide error: 0000 [#2] SMP [20918.017044] Modules linked in: binfmt_misc(F) cmtp(F) kernelcapi(F) tun(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) bnep(F) hidp(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) rds(F) af_key(F) bluetooth(F) rfkill(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) nfc(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) bnx2(F) kvm_amd(F) amd64_edac_mod(F) edac_mce_amd(F) kvm(F) pcspkr(F) serio_raw(F) shpchp(F) i2c_piix4(F) edac_core(F) k8temp(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) radeon(F) i2c_algo_bit(F) drm_kms_helper(F) mptsas(F) ttm(F) scsi_transport_sas(F) drm(F ) mptscsih(F) i2c_core(F) mptbase(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [20918.017044] CPU 2 [20918.017044] Pid: 0, comm: swapper/2 Tainted: GF D 3.9.4 #1 IBM BladeCenter LS41 -[7972AC1]-/Server Blade [20918.017044] RIP: 0010:[<ffffffff8109ed2b>] [<ffffffff8109ed2b>] update_sd_lb_stats+0x2bb/0x5b0 [20918.017044] RSP: 0018:ffff88033bc03c00 EFLAGS: 00010216 [20918.017044] RAX: 0000000000040800 RBX: 0000000000000000 RCX: ffff880439f5ba18 [20918.017044] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000008 [20918.017044] RBP: ffff88033bc03cc8 R08: 0000000000000102 R09: 0000000000000005 [20918.017044] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000008 [20918.017044] R13: 0000000000000000 R14: 0000000000000001 R15: ffff88033bc03df0 [20918.017044] FS: 00007fb8dd06e740(0000) GS:ffff88033bc00000(0000) knlGS:0000000000000000 [20918.017044] CS: 0010 DS: 0000 ES: 0000 CR[-- MARK -- Thu May 30 10:40:00 2013] 3) [10721.161888] XFS (dm-0): Ending clean mount [10722.090855] XFS (dm-2): Mounting Filesystem [10722.143682] XFS (dm-2): Ending clean mount [10722.584327] XFS (dm-0): Mounting Filesystem [10722.949519] XFS (dm-0): Ending clean mount [10723.579704] BUG: unable to handle kernel paging request at ffff8801b55cc000 [10723.619185] IP: [<ffffffff81302305>] memmove+0x55/0x1a0 [10723.647690] PGD 1ddf067 PUD 20bdfa067 PMD 1b5734063 PTE 80000001b55cc161 [10723.685013] Oops: 0003 [#1] SMP [10723.701968] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F)[10724.12 8502] sleep[6604]: segfault at 0 ip (null) sp 00007fff96b37a30 error 14 in sleep[400000+6000] [10724.128757] ------------[ cut here ]------------ [10724.128760] WARNING: at mm/mmap.c:2711 exit_mmap+0x166/0x170() [10724.128761] Hardware name: ProLiant DL120 G7 [10724.128761] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10724.128795] Pid: 6604, comm: sleep Tainted: GF 3.9.4 #1 [10724.128796] Call Trace: [10724.128796] [<ffffffff81060f50>] warn_slowpath_common+0x70/0xa0 [10724.128800] traps: auditd[520] general protection ip:7f9a89363be5 sp:7f9a8801f680 error:0 [<ffffffff8106103a>] warn_slowpath_null+0x1a/0x20 [10724.128802] [<ffffffff811647d6>] exit_mmap+0x166/0x170 [10724.128804] in libc-2.17.so[7f9a8931c000+1b5000] [<ffffffff8105e377>] mmput+0x67/0xf0 [10724.128806] [10724.128806] [<ffffffff81066898>] do_exit+0x278/0xa20 [10724.128816] [<ffffffff810670bf>] do_group_exit+0x3f/0xa0 [10724.128817] [<ffffffff81075ead>] get_signal_to_deliver+0x1ad/0x5c0 [10724.128820] [<ffffffff81013438>] do_signal+0x48/0x5a0 [10724.128823] [<ffffffff81614d4c>] ? __do_page_fault+0x45c/0x510 [10724.128825] [<ffffffff810879b8>] ? hrtimer_try_to_cancel+0x48/0xe0 [10724.128827] [<ffffffff8108879d>] ? hrtimer_nanosleep+0xad/0x170 [10724.128829] [<ffffffff81087820>] ? hrtimer_get_res+0x50/0x50 [10724.128830] [<ffffffff81013a00>] do_notify_resume+0x70/0xa0 [10724.128831] [<ffffffff816112bc>] retint_signal+0x48/0x8c [10724.128834] ---[ end trace aa0156eab31509fc ]--- [10724.128901] BUG: Bad rss-counter state mm:ffff8801fb90af40 idx:0 val:5 [10724.128901] BUG: Bad rss-counter state mm:ffff8801fb90af40 idx:1 val:3 [10724.128927] rhts-test-runne[48621]: segfault at 19cc ip 00000000004401c4 sp 00007fff27f840e0 error 6 in bash[400000+db000] [10724.129726] ============================================================================= [10724.129727] BUG kmalloc-4096 (Tainted: GF W ): Padding overwritten. 0xffff88019152fc28-0xffff88019152ffff [10724.129727] ----------------------------------------------------------------------------- [10724.129727] [10724.129728] INFO: Slab 0xffffea0006454a00 objects=7 used=7 fp=0x (null) flags=0x20000000004080 [10724.129729] Pid: 43729, comm: kworker/0:0 Tainted: GF B W 3.9.4 #1 [10724.129729] Call Trace: [10724.129730] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 [10724.129733] [<ffffffff812f8007>] ? kset_init+0x27/0x40 [10724.129735] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 [10724.129736] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 [10724.129739] [<ffffffff811820e3>] check_slab+0x73/0x100 [10724.129740] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 [10724.129742] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 [10724.129744] [<ffffffff8109f5d8>] ? load_balance+0x108/0x7d0 [10724.129746] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 [10724.129747] [<ffffffff8108fa7c>] ? update_rq_clock.part.67+0x1c/0x170 [10724.129749] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 [10724.129751] [<ffffffff810ce4d2>] cgroup_release_agent+0x42/0x180 [10724.129752] [<ffffffff8160eda0>] ? __schedule+0x3b0/0x7d0 [10724.129754] [<ffffffff8107d80f>] process_one_work+0x16f/0x3f0 [10724.129756] [<ffffffff8107f360>] worker_thread+0x110/0x3d0 [10724.129757] [<ffffffff8107f250>] ? __alloc_workqueue_key+0x500/0x500 [10724.129758] [<ffffffff81084240>] kthread+0xc0/0xd0 [10724.129759] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10724.129760] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [10724.129762] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10724.129765] Padding ffff88019152f8f8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129765] Padding ffff88019152f908: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129766] Padding ffff88019152f918: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129766] Padding ffff88019152f928: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129767] Padding ffff88019152f938: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129767] Padding ffff88019152f948: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129768] Padding ffff88019152f958: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129768] Padding ffff88019152f968: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129769] Padding ffff88019152f978: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129769] Padding ffff88019152f988: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129770] Padding ffff88019152f998: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129770] Padding ffff88019152f9a8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129771] Padding ffff88019152f9b8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129771] Padding ffff88019152f9c8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129772] Padding ffff88019152f9d8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129772] Padding ffff88019152f9e8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129773] Padding ffff88019152f9f8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129773] Padding ffff88019152fa08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129774] Padding ffff88019152fa18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129774] Padding ffff88019152fa28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129775] Padding ffff88019152fa38: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129775] Padding ffff88019152fa48: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129776] Padding ffff88019152fa58: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129776] Padding ffff88019152fa68: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129777] Padding ffff88019152fa78: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129777] Padding ffff88019152fa88: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129778] Padding ffff88019152fa98: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129778] Padding ffff88019152faa8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129779] Padding ffff88019152fab8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129779] Padding ffff88019152fac8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129780] Padding ffff88019152fad8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129780] Padding ffff88019152fae8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129781] Padding ffff88019152faf8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129781] Padding ffff88019152fb08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129782] Padding ffff88019152fb18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129782] Padding ffff88019152fb28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129783] Padding ffff88019152fb38: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129783] Padding ffff88019152fb48: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129784] Padding ffff88019152fb58: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129784] Padding ffff88019152fb68: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129785] Padding ffff88019152fb78: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129785] Padding ffff88019152fb88: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129786] Padding ffff88019152fb98: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129786] Padding ffff88019152fba8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129787] Padding ffff88019152fbb8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129787] Padding ffff88019152fbc8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129788] Padding ffff88019152fbd8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129788] Padding ffff88019152fbe8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129789] Padding ffff88019152fbf8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129789] Padding ffff88019152fc08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129790] Padding ffff88019152fc18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [10724.129790] Padding ffff88019152fc28: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129791] Padding ffff88019152fc38: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129792] Padding ffff88019152fc48: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129792] Padding ffff88019152fc58: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129793] Padding ffff88019152fc68: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129793] Padding ffff88019152fc78: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129794] Padding ffff88019152fc88: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129794] Padding ffff88019152fc98: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129795] Padding ffff88019152fca8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129795] Padding ffff88019152fcb8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129796] Padding ffff88019152fcc8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129796] Padding ffff88019152fcd8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129797] Padding ffff88019152fce8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129797] Padding ffff88019152fcf8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129798] Padding ffff88019152fd08: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129798] Padding ffff88019152fd18: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129799] Padding ffff88019152fd28: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129799] Padding ffff88019152fd38: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129800] Padding ffff88019152fd48: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129800] Padding ffff88019152fd58: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129801] Padding ffff88019152fd68: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129801] Padding ffff88019152fd78: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129802] Padding ffff88019152fd88: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129802] Padding ffff88019152fd98: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129803] Padding ffff88019152fda8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129803] Padding ffff88019152fdb8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129804] Padding ffff88019152fdc8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129804] Padding ffff88019152fdd8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129805] Padding ffff88019152fde8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129805] Padding ffff88019152fdf8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129806] Padding ffff88019152fe08: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129806] Padding ffff88019152fe18: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129807] Padding ffff88019152fe28: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129807] Padding ffff88019152fe38: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129808] Padding ffff88019152fe48: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129808] Padding ffff88019152fe58: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129809] Padding ffff88019152fe68: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129809] Padding ffff88019152fe78: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129810] Padding ffff88019152fe88: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129810] Padding ffff88019152fe98: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129811] Padding ffff88019152fea8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129811] Padding ffff88019152feb8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129812] Padding ffff88019152fec8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129812] Padding ffff88019152fed8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129813] Padding ffff88019152fee8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129813] Padding ffff88019152fef8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129814] Padding ffff88019152ff08: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129814] Padding ffff88019152ff18: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129815] Padding ffff88019152ff28: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129815] Padding ffff88019152ff38: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129816] Padding ffff88019152ff48: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129816] Padding ffff88019152ff58: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129817] Padding ffff88019152ff68: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129817] Padding ffff88019152ff78: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129818] Padding ffff88019152ff88: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129818] Padding ffff88019152ff98: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129819] Padding ffff88019152ffa8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129819] Padding ffff88019152ffb8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129820] Padding ffff88019152ffc8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129820] Padding ffff88019152ffd8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129821] Padding ffff88019152ffe8: c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 c6 ................ [10724.129821] Padding ffff88019152fff8: c6 c6 c6 c6 c6 c6 c6 c6 ........ [10724.129822] FIX kmalloc-4096: Restoring 0xffff88019152f8f8-0xffff88019152ffff=0x5a [10724.129822] [10724.135254] systemd[1]: segfault at 45 ip 00007f93e348c780 sp 00007fff4deb3a9e error 4 in libdbus-1.so.3.7.2[7f93e3479000+45000] [10724.135272] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb34f8 error:0 in systemd[7f93e496d000+f6000] [10724.135281] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb2f38 error:0 in systemd[7f93e496d000+f6000] [10724.135289] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb2978 error:0 in systemd[7f93e496d000+f6000] [10724.135297] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb23b8 error:0 in systemd[7f93e496d000+f6000] [10724.135304] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb1df8 error:0 in systemd[7f93e496d000+f6000] [10724.135312] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb1838 error:0 in systemd[7f93e496d000+f6000] [10724.169488] audit: *NO* daemon at audit_pid=505 [10732.129639] i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10732.206898] CPU 3 [10732.215891] Pid: 6911, comm: attr Tainted: GF B W 3.9.4 #1 HP ProLiant DL120 G7 [10732.256496] RIP: 0010:[<ffffffff81302305>] [<ffffffff81302305>] memmove+0x55/0x1a0 [10732.296932] RSP: 0018:ffff8801bbe15ad8 EFLAGS: 00010282 [10732.325205] RAX: ffff880191529168 RBX: ffff8801e687b000 RCX: 000000000000d00b [10732.363603] RDX: ffffffffdbf5d138 RSI: ffff8801b55cc3e0 RDI: ffff8801b55cbfe8 [10732.401581] RBP: ffff8801bbe15b30 R08: 38408bffffff4085 R09: 8b48dceb000000b8 [10732.440757] R10: 9f89498eeb000000 R11: 20befffffc65e9e2 R12: ffff880191529148 [10732.475959] R13: 00000000000003d8 R14: 0000000000001800 R15: 000000000000007b [10732.512079] FS: 00007fc626dc9740(0000) GS:ffff88020b460000(0000) knlGS:0000000000000000 [10732.553296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10732.581757] CR2: ffff8801b55cc000 CR3: 00000001e169c000 CR4: 00000000000407e0 [10732.617759] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10732.653908] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10732.689045] Process attr (pid: 6911, threadinfo ffff8801bbe14000, task ffff8801e6b71ac0) [10732.729564] Stack: [10732.739413] ffffffffa01651a1 0000000000000000 0000000000000000 ffff880191529168 [10732.776436] ffff880191529148 0000000000000018 ffff880191529148 0000000000000000 [10732.813437] ffff8801de8237b0 ffff8801c491d280 ffff8801e2c776c0 ffff8801bbe15b68 [10732.852979] Call Trace: [10732.865992] [<ffffffffa01651a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [10732.906399] [<ffffffffa0165467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [10732.942002] [<ffffffffa0166a2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [10732.975905] [<ffffffffa0162850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [10733.009452] [<ffffffffa0170d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [10733.042797] [<ffffffffa016378c>] xfs_attr_set_int+0x30c/0x420 [xfs] [10733.074645] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [10733.100539] [<ffffffffa0163d1f>] xfs_attr_set+0x7f/0x90 [xfs] [10733.129870] [<ffffffffa0159a12>] xfs_xattr_set+0x42/0x70 [xfs] [10733.159595] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [10733.188208] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [10733.218633] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [10733.245380] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [10733.270691] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [10733.300416] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [10733.327832] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [10733.354383] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [10733.385198] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [10733.416587] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [10733.445421] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [10733.477008] Code: 40 38 fe 74 41 48 83 ea 20 48 83 ea 20 4c 8b 1e 4c 8b 56 08 4c 8b 4e 10 4c 8b 46 18 48 8d 76 20 4c 89 1f 4c 89 57 08 4c 89 4f 10 <4c> 89 47 18 48 8d 7f 20 73 d4 48 83 c2 20 e9 a8 00 00 00 0f 1f [10733.575487] RIP [<ffffffff81302305>] memmove+0x55/0x1a0 [10733.602350] RSP <ffff8801bbe15ad8> [10733.620195] CR2: ffff8801b55cc000 [10733.693452] ---[ end trace aa0156eab31509fd ]--- [10733.693454] general protection fault: 0000 [#2] SMP [10733.693468] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10733.693475] CPU 1 [10733.693475] Pid: 48621, comm: rhts-test-runne Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 [10733.693478] RIP: 0010:[<ffffffff8115787e>] [<ffffffff8115787e>] vma_interval_tree_remove+0x3e/0x240 [10733.693479] RSP: 0000:ffff8801fcdcfba0 EFLAGS: 00010206 [10733.693479] RAX: 0000be0f00000000 RBX: ffff8801e0abda00 RCX: ffff8801a65ca058 [10733.693480] RDX: 0000be0f00000000 RSI: ffff8801f74b0760 RDI: ffff8801e0abda00 [10733.693480] RBP: ffff8801fcdcfbb8 R08: ffff8801a65cb800 R09: 0000000000017700 [10733.693481] R10: ffff8801e0abda58 R11: 000000000000000e R12: ffff8801e0abda00 [10733.693481] R13: ffff8801f74b0778 R14: ffff8801f74b0740 R15: ffff8801e0abda00 [10733.693482] FS: 0000000000000000(0000) GS:ffff88020b420000(0000) knlGS:0000000000000000 [10733.693483] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10733.693483] CR2: 00000000000019cc CR3: 00000001fbc0b000 CR4: 00000000000407e0 [10733.693484] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10733.693484] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10733.693485] Process rhts-test-runne (pid: 48621, threadinfo ffff8801fcdce000, task ffff8801e54f3580) [10733.693485] Stack: [10733.693487] ffffffff811613e6 ffff8801c4a68480 ffff8801e0abda00 ffff8801fcdcfbe8 [10733.693488] ffffffff811619a0 ffff8801dc1c1000 00007f9810be3000 0000000000000000 [10733.693489] ffff8801fcdcfc40 ffff8801fcdcfc28 ffffffff8115a240 0000000000000000 [10733.693489] Call Trace: [10733.693491] [<ffffffff811613e6>] ? __remove_shared_vm_struct+0x46/0x70 [10733.693492] [<ffffffff811619a0>] unlink_file_vma+0x40/0x60 [10733.693493] [<ffffffff8115a240>] free_pgtables+0xb0/0x120 [10733.693495] [<ffffffff81164722>] exit_mmap+0xb2/0x170 [10733.693498] [<ffffffff8105e377>] mmput+0x67/0xf0 [10733.693500] [<ffffffff81066898>] do_exit+0x278/0xa20 [10733.693501] [<ffffffff810670bf>] do_group_exit+0x3f/0xa0 [10733.693503] [<ffffffff81075ead>] get_signal_to_deliver+0x1ad/0x5c0 [10733.693506] [<ffffffff81013438>] do_signal+0x48/0x5a0 [10733.693507] [<ffffffff81614d4c>] ? __do_page_fault+0x45c/0x510 [10733.693510] [<ffffffff8130aabd>] ? list_del+0xd/0x30 [10733.693511] [<ffffffff81084f51>] ? remove_wait_queue+0x31/0x40 [10733.693513] [<ffffffff8106736e>] ? sys_wait4+0x6e/0xe0 [10733.693514] [<ffffffff81013a00>] do_notify_resume+0x70/0xa0 [10733.693516] [<ffffffff816112bc>] retint_signal+0x48/0x8c [10733.693525] Code: d2 0f 84 9f 01 00 00 48 85 c9 0f 84 bd 01 00 00 48 8b 41 10 49 89 c8 48 85 c0 75 0f e9 cb 01 00 00 0f 1f 40 00 49 89 c0 48 89 d0 <48> 8b 50 10 48 85 d2 75 f1 4c 8b 58 08 4d 89 58 10 48 89 48 08 [10733.693526] RIP [<ffffffff8115787e>] vma_interval_tree_remove+0x3e/0x240 [10733.693527] RSP <ffff8801fcdcfba0> [10733.693529] general protection fault: 0000 [#3] SMP [10733.693549] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10733.693557] CPU 0 [10733.693557] Pid: 43729, comm: kworker/0:0 Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 [10733.693559] RIP: 0010:[<ffffffff81183889>] [<ffffffff81183889>] deactivate_slab+0x119/0x510 [10733.693562] RSP: 0000:ffff8801fdfd5bb8 EFLAGS: 00010006 [10733.693563] RAX: 0000000000001008 RBX: ffff88019152e7b0 RCX: 0000000180070006 [10733.693564] RDX: 0000000180070007 RSI: ffffea0006454a00 RDI: ffff880202c0ee40 [10733.693565] RBP: ffff8801fdfd5ca8 R08: 0000000000000000 R09: ffff880191529000 [10733.693567] R10: 0000000000000000 R11: ffff880202c00c40 R12: ffffea0006454a00 [10733.693567] R13: ffff880202c0ee40 R14: 5a5a5a5a5a5a5a5a R15: 00000000ffffffff [10733.693568] FS: 0000000000000000(0000) GS:ffff88020b400000(0000) knlGS:0000000000000000 [10733.693569] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10733.693569] CR2: 0000000000000000 CR3: 00000000018fa000 CR4: 00000000000407f0 [10733.693570] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10733.693570] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10733.693572] Process kworker/0:0 (pid: 43729, threadinfo ffff8801fdfd4000, task ffff8801b575d040) [10733.693572] Stack: [10733.693576] ffffffff810154cf ffff8801fdfd5bf4 0000000f00000001 ffff88020b403fc0 [10733.693578] 0000000000000000 ffff8801fdfd4000 ffff8801fdfd5c50 00000000811817db [10733.693580] ffffea0006454a00 ffff8801fdfd5c50 ffffffff810ce4d2 ffff880191528000 [10733.693581] Call Trace: [10733.693584] [<ffffffff810154cf>] ? dump_trace+0x16f/0x2b0 [10733.693586] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 [10733.693589] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [10733.693590] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [10733.693592] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [10733.693593] ---[ end trace aa0156eab31509fe ]--- [10733.693595] [<ffffffff81607756>] __slab_alloc+0x3df/0x4a2 [10733.693595] Fixing recursive fault but reboot is needed! [10733.693597] [<ffffffff8109f5d8>] ? load_balance+0x108/0x7d0 [10733.693599] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 [10733.693601] [<ffffffff8108fa7c>] ? update_rq_clock.part.67+0x1c/0x170 [10733.693603] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 [10733.693605] [<ffffffff810ce4d2>] cgroup_release_agent+0x42/0x180 [10733.693607] [<ffffffff8160eda0>] ? __schedule+0x3b0/0x7d0 [10733.693610] [<ffffffff8107d80f>] process_one_work+0x16f/0x3f0 [10733.693612] [<ffffffff8107f360>] worker_thread+0x110/0x3d0 [10733.693613] [<ffffffff8107f250>] ? __alloc_workqueue_key+0x500/0x500 [10733.693615] [<ffffffff81084240>] kthread+0xc0/0xd0 [10733.693616] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10733.693618] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [10733.693620] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10733.693642] Code: 40 00 49 3b 54 24 18 75 ed 49 89 5c 24 10 49 89 4c 24 18 41 0f ba 34 24 00 66 0f 1f 84 00 00 00 00 00 4d 85 f6 74 1d 49 63 45 20 <49> 8b 14 06 48 85 d2 74 10 4c 89 f3 4d 8b 44 24 10 49 89 d6 e9 [10733.693643] RIP [<ffffffff81183889>] deactivate_slab+0x119/0x510 [10733.693644] RSP <ffff8801fdfd5bb8> [10733.693645] ---[ end trace aa0156eab31509ff ]--- [10733.693673] audit_printk_skb: 5695 callbacks suppressed [10733.693674] type=1701 audit(1369899862.224:50838): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:syslogd_t:s0 pid=38988 comm="in:imklog" reason="memory violation" sig=11 [10733.693771] general protection fault: 0000 [#4] SMP [10733.693783] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10733.693790] CPU 2 [10733.693790] Pid: 38988, comm: in:imklog Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 [10733.693792] RIP: 0010:[<ffffffff810c8b00>] [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [10733.693793] RSP: 0000:ffff8801e7977cd0 EFLAGS: 00010206 [10733.693793] RAX: 0000be0d00000000 RBX: ffff8801deb93c80 RCX: ffff8801e7977fd8 [10733.693794] RDX: 0000000000021000 RSI: 0000000000000001 RDI: ffff8801de900de8 [10733.693794] RBP: ffff8801e7977cf0 R08: ffff8801ff629ac0 R09: 0000000000000000 [10733.693795] R10: 0000000000000000 R11: 0000000000000001 R12: ffff8801ff629ac0 [10733.693795] R13: 000000000000000b R14: 0000000000000000 R15: ffff8801ff629ac0 [10733.693796] FS: 00007fc4aade0700(0000) GS:ffff88020b440000(0000) knlGS:0000000000000000 [10733.693797] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10733.693797] CR2: 00007fff7fbabb10 CR3: 000000017e40c000 CR4: 00000000000407e0 [10733.693798] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10733.693799] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10733.693799] Process in:imklog (pid: 38988, threadinfo ffff8801e7976000, task ffff8801ff629ac0) [10733.693800] Stack: [10733.693801] 000000000000000b 000000000000984c ffff8801ff629ac0 ffff8801ff629ac0 [10733.693802] ffff8801e7977d78 ffffffff81066e01 00000001de900de8 ffff8801f9ee1128 [10733.693803] ffff8801e193ee50 000000800000000a ffff8801e7977e78 ffff8801e7977f58 [10733.693803] Call Trace: [10733.693805] [<ffffffff81066e01>] do_exit+0x7e1/0xa20 [10733.693807] [<ffffffff810670bf>] do_group_exit+0x3f/0xa0 [10733.693808] [<ffffffff81075ead>] get_signal_to_deliver+0x1ad/0x5c0 [10733.693810] [<ffffffff81013438>] do_signal+0x48/0x5a0 [10733.693812] [<ffffffff81614d4c>] ? __do_page_fault+0x45c/0x510 [10733.693813] [<ffffffff810850e0>] ? wake_up_bit+0x30/0x30 [10733.693814] [<ffffffff81013a00>] do_notify_resume+0x70/0xa0 [10733.693816] [<ffffffff816112bc>] retint_signal+0x48/0x8c [10733.693826] Code: 00 00 00 74 56 49 8b 84 24 80 04 00 00 48 8d 78 68 e8 25 5b 54 00 49 8b 84 24 80 04 00 00 48 8b 00 48 85 c0 74 1c 31 d2 0f 1f 00 <48> 03 50 08 48 2b 10 48 8b 40 10 48 85 c0 75 f0 48 c1 ea 0a 49 [10733.693827] RIP [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [10733.693827] RSP <ffff8801e7977cd0> [10733.693828] ---[ end trace aa0156eab3150a00 ]--- [10733.693828] Fixing recursive fault but reboot is needed! [10733.694251] NetworkManager[568]: segfault at 75 ip 00007eff6bc52bcb sp 00007fff5e0d5db0 error 4 in libglib-2.0.so.0.3600.1[7eff6bc0b000+12a000] [10733.694275] type=1701 audit(1369899862.225:50839): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:NetworkManager_t:s0 pid=568 comm="NetworkManager" reason="memory violation" sig=11 [10733.695784] BUG: unable to handle kernel paging request at ffffffffffffffd8 [10733.695786] IP: [<ffffffff810847e0>] kthread_data+0x10/0x20 [10733.695787] PGD 18fd067 PUD 18ff067 PMD 0 [10733.695788] Oops: 0000 [#5] SMP [10733.695799] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10733.695806] CPU 0 [10733.695806] Pid: 43729, comm: kworker/0:0 Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 [10733.695807] RIP: 0010:[<ffffffff810847e0>] [<ffffffff810847e0>] kthread_data+0x10/0x20 [10733.695808] RSP: 0018:ffff8801fdfd5968 EFLAGS: 00010002 [10733.695808] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000f [10733.695809] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8801b575d040 [10733.695809] RBP: ffff8801fdfd5968 R08: ffff8801b575d0b0 R09: 000009c3221f4808 [10733.695810] R10: 0000000000000000 R11: ffffea0007952900 R12: ffff88020b414400 [10733.695810] R13: 0000000000000000 R14: ffff8801b575d030 R15: ffff8801b575d040 [10733.695811] FS: 0000000000000000(0000) GS:ffff88020b400000(0000) knlGS:0000000000000000 [10733.695812] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10733.695812] CR2: ffffffffffffffd8 CR3: 00000001e784d000 CR4: 00000000000407f0 [10733.695813] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10733.695813] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10733.695814] Process kworker/0:0 (pid: 43729, threadinfo ffff8801fdfd4000, task ffff8801b575d040) [10733.695814] Stack: [10733.695815] ffff8801fdfd5980 ffffffff8107f9a1 ffff8801b575d040 ffff8801fdfd59e0 [10733.695816] ffffffff8160efa2 ffff8801fdfd5fd8 0000000000014400 ffff8801fdfd5fd8 [10733.695817] 0000000000014400 ffff8801b575d040 ffff8801b575d828 ffff8801fdfd5550 [10733.695818] Call Trace: [10733.695819] [<ffffffff8107f9a1>] wq_worker_sleeping+0x11/0x70 [10733.695821] [<ffffffff8160efa2>] __schedule+0x5b2/0x7d0 [10733.695822] [<ffffffff8160f1e9>] schedule+0x29/0x70 [10733.695823] [<ffffffff81066ce7>] do_exit+0x6c7/0xa20 [10733.695825] [<ffffffff81612034>] oops_end+0xa4/0xe0 [10733.695827] [<ffffffff810168db>] die+0x4b/0x70 [10733.695828] [<ffffffff81611ada>] do_general_protection+0xda/0x160 [10733.695830] [<ffffffff81611468>] general_protection+0x28/0x30 [10733.695832] [<ffffffff81183889>] ? deactivate_slab+0x119/0x510 [10733.695833] [<ffffffff810154cf>] ? dump_trace+0x16f/0x2b0 [10733.695835] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 [10733.695837] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [10733.695838] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [10733.695839] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [10733.695840] [<ffffffff81607756>] __slab_alloc+0x3df/0x4a2 [10733.695842] [<ffffffff8109f5d8>] ? load_balance+0x108/0x7d0 [10733.695843] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 [10733.695844] [<ffffffff8108fa7c>] ? update_rq_clock.part.67+0x1c/0x170 [10733.695846] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 [10733.695847] [<ffffffff810ce4d2>] cgroup_release_agent+0x42/0x180 [10733.695848] [<ffffffff8160eda0>] ? __schedule+0x3b0/0x7d0 [10733.695849] [<ffffffff8107d80f>] process_one_work+0x16f/0x3f0 [10733.695851] [<ffffffff8107f360>] worker_thread+0x110/0x3d0 [10733.695852] [<ffffffff8107f250>] ? __alloc_workqueue_key+0x500/0x500 [10733.695853] [<ffffffff81084240>] kthread+0xc0/0xd0 [10733.695854] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10733.695855] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [10733.695856] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10733.695865] Code: 00 48 89 e5 5d 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 80 05 00 00 55 48 89 e5 <48> 8b 40 d8 5d c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 [10733.695866] RIP [<ffffffff810847e0>] kthread_data+0x10/0x20 [10733.695867] RSP <ffff8801fdfd5968> [10733.695867] CR2: ffffffffffffffd8 [10733.695868] ---[ end trace aa0156eab3150a01 ]--- [10733.695868] Fixing recursive fault but reboot is needed! [10741.654996] BUG: unable to handle kernel paging request at 00007f18c4071018 [10741.691831] IP: [<ffffffff8115798b>] vma_interval_tree_remove+0x14b/0x240 [10741.727684] PGD 0 [10741.738338] Oops: 0000 [#6] SMP [10741.755746] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10742.257869] CPU 3 [10742.267698] Pid: 6911, comm: attr Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 [10742.311646] RIP: 0010:[<ffffffff8115798b>] [<ffffffff8115798b>] vma_interval_tree_remove+0x14b/0x240 [10742.358495] RSP: 0018:ffff8801bbe15670 EFLAGS: 00010206 [10742.384615] RAX: ffff8801a65cae20 RBX: ffff8801fb849200 RCX: 00007f18c4071000 [10742.420770] RDX: ffffffffffffffff RSI: ffff8801feb63220 RDI: ffff8801fb849200 [10742.456593] RBP: ffff8801bbe15688 R08: ffff8801fba5dc58 R09: ffff8801fb849a58 [10742.494108] R10: ffff8801fb849258 R11: 0000000000000000 R12: ffff8801fb849200 [10742.530422] R13: ffff8801feb63238 R14: ffff8801feb63200 R15: ffff8801fb849200 [10742.565618] FS: 0000000000000000(0000) GS:ffff88020b460000(0000) knlGS:0000000000000000 [10742.606623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10742.635672] CR2: 00007f18c4071018 CR3: 00000001e169c000 CR4: 00000000000407e0 [10742.670958] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10742.706707] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10742.744778] Process attr (pid: 6911, threadinfo ffff8801bbe14000, task ffff8801e6b71ac0) [10742.788295] Stack: [10742.798965] ffffffff811613e6 ffff8801c46e7740 ffff8801fb849200 ffff8801bbe156b8 [10742.839037] ffffffff811619a0 ffff8801fb849a00 00007fc6265f1000 0000000000000000 [10742.876283] ffff8801bbe15710 ffff8801bbe156f8 ffffffff8115a240 0000000000000000 [10742.913418] Call Trace: [10742.925426] [<ffffffff811613e6>] ? __remove_shared_vm_struct+0x46/0x70 [10742.958588] [<ffffffff811619a0>] unlink_file_vma+0x40/0x60 [10742.985945] [<ffffffff8115a240>] free_pgtables+0xb0/0x120 [10743.013519] [<ffffffff81164722>] exit_mmap+0xb2/0x170 [10743.038820] [<ffffffff8105e377>] mmput+0x67/0xf0 [10743.062990] [<ffffffff81066898>] do_exit+0x278/0xa20 [10743.087864] [<ffffffff81604989>] ? printk+0x67/0x69 [10743.112837] [<ffffffff81063088>] ? console_unlock+0x1e8/0x3f0 [10743.141614] [<ffffffff81612034>] oops_end+0xa4/0xe0 [10743.166575] [<ffffffff81604212>] no_context+0x263/0x270 [10743.192871] [<ffffffff81604292>] __bad_area_nosemaphore+0x73/0x1cc [10743.224652] [<ffffffff816043fe>] bad_area_nosemaphore+0x13/0x15 [10743.255236] [<ffffffff81614c0e>] __do_page_fault+0x31e/0x510 [10743.286559] [<ffffffffa015d497>] ? kmem_alloc+0x67/0xf0 [xfs] [10743.317700] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [10743.349363] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [10743.378060] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [10743.407085] [<ffffffff81614e0e>] do_page_fault+0xe/0x10 [10743.436241] [<ffffffff81611498>] page_fault+0x28/0x30 [10743.462402] [<ffffffff81302305>] ? memmove+0x55/0x1a0 [10743.487998] [<ffffffffa01651a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [10743.526426] [<ffffffffa0165467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [10743.559879] [<ffffffffa0166a2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [10743.591996] [<ffffffffa0162850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [10743.625676] [<ffffffffa0170d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [10743.658136] [<ffffffffa016378c>] xfs_attr_set_int+0x30c/0x420 [xfs] [10743.689678] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [10743.716002] [<ffffffffa0163d1f>] xfs_attr_set+0x7f/0x90 [xfs] [10743.744715] [<ffffffffa0159a12>] xfs_xattr_set+0x42/0x70 [xfs] [10743.774512] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [10743.802374] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [10743.833373] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [10743.859695] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [10743.887070] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [10743.919013] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [10743.948596] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [10743.976310] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [10744.007107] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [10744.036839] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [10744.063840] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [10744.094150] Code: 48 89 50 18 48 8b 00 48 83 e0 fc 74 43 48 8b 50 b0 48 2b 50 a8 48 8b 48 40 48 c1 ea 0c 48 8d 54 11 ff 48 8b 48 10 48 85 c9 74 0b <48> 8b 49 18 48 39 ca 48 0f 42 d1 48 8b 48 08 48 85 c9 74 0b 48 [10744.188945] RIP [<ffffffff8115798b>] vma_interval_tree_remove+0x14b/0x240 [10744.223099] RSP <ffff8801bbe15670> [10744.240096] CR2: 00007f18c4071018 [10744.257118] ---[ end trace aa0156eab3150a02 ]--- [10744.280164] Fixing recursive fault but reboot is needed! [10748.689674] general protection fault: 0000 [#7] SMP [10748.716550] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10749.217082] CPU 2 [10749.227162] Pid: 464, comm: flush-253:1 Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 [10749.274289] RIP: 0010:[<ffffffff81157609>] [<ffffffff81157609>] vma_interval_tree_subtree_search+0x19/0x80 [10749.326848] RSP: 0000:ffff8801e6937a30 EFLAGS: 00010202 [10749.355371] RAX: 6b6b6b6b6b6b6b6b RBX: ffff8801f74a6fc0 RCX: 0000000000000276 [10749.394358] RDX: 00000000000006af RSI: 00000000000006af RDI: ffff8801a65c8400 [10749.429501] RBP: ffff8801e6937a30 R08: ffff8801e6937b01 R09: 0000000000001000 [10749.465183] R10: 0000000000000100 R11: 0000000000000210 R12: 00000000000006af [10749.500930] R13: ffff8801e6937cc8 R14: ffffea00047e1340 R15: ffffea00047e1340 [10749.536294] FS: 0000000000000000(0000) GS:ffff88020b440000(0000) knlGS:0000000000000000 [10749.577185] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10749.605877] CR2: 0000000000000000 CR3: 00000000018fa000 CR4: 00000000000407e0 [10749.641518] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10749.677263] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10749.712490] Process flush-253:1 (pid: 464, threadinfo ffff8801e6936000, task ffff8801fcea3580) [10749.755984] Stack: [10749.765953] ffff8801e6937a40 ffffffff81157aa2 ffff8801e6937a98 ffffffff8116714e [10749.803931] ffff8801f74a6ff8 00000100638e3f00 00000000000006b1 00000000000006c0 [10749.843909] ffffea00047e1340 ffff8801f74a6fc0 ffff8801e6937cc8 0000000000000000 [10749.882823] Call Trace: [10749.895832] [<ffffffff81157aa2>] vma_interval_tree_iter_first+0x22/0x30 [10749.932506] [<ffffffff8116714e>] page_mkclean+0x6e/0x1d0 [10749.960173] [<ffffffff8113dbb8>] clear_page_dirty_for_io+0x48/0xc0 [10749.991604] [<ffffffff8113de4a>] write_cache_pages+0x21a/0x4b0 [10750.020918] [<ffffffff8113d4b0>] ? global_dirtyable_memory+0x50/0x50 [10750.053649] [<ffffffff8113e120>] generic_writepages+0x40/0x60 [10750.083106] [<ffffffffa01449e5>] xfs_vm_writepages+0x35/0x40 [xfs] [10750.113998] [<ffffffff8113f02e>] do_writepages+0x1e/0x40 [10750.141300] [<ffffffff811c2ec0>] __writeback_single_inode+0x40/0x210 [10750.173209] [<ffffffff811c4b77>] writeback_sb_inodes+0x197/0x400 [10750.204613] [<ffffffff811c4e7f>] __writeback_inodes_wb+0x9f/0xd0 [10750.235080] [<ffffffff811c5543>] wb_writeback+0x233/0x2b0 [10750.263128] [<ffffffff811c6f65>] wb_do_writeback+0x1e5/0x1f0 [10750.291413] [<ffffffff811c6ffb>] bdi_writeback_thread+0x8b/0x210 [10750.321825] [<ffffffff811c6f70>] ? wb_do_writeback+0x1f0/0x1f0 [10750.35 80>] ? insert_kthread_work+0x40/0x40 [10750.409867] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [10750.439257] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10750.471505] Code: c3 0f 0b 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 66 66 66 66 90 55 48 89 e5 0f 1f 80 00 00 00 00 48 8b 47 68 48 85 c0 74 17 <48> 39 70 18 72 11 48 8d 78 a8 48 8b 47 68 48 85 c0 75 ed 0f 1f [10750.568513] RIP [<ffffffff81157609>] vma_interval_tree_subtree_search+0x19/0x80 [10750.605629] RSP <ffff8801e6937a30> [10750.624263] ---[ end trace aa0156eab3150a03 ]--- [10761.391486] general protection fault: 0000 [#8] SMP [10761.418069] Modules linked in: binfmt_misc(F) bnep(F) hidp(F) tun(F) fuse(F) cmtp(F) kernelcapi(F) l2tp_ppp(F) l2tp_netlink(F) l2tp_core(F) rfcomm(F) nfnetlink(F) scsi_transport_iscsi(F) ipt_ULOG(F) af_802154(F) atm(F) bluetooth(F) rfkill(F) af_key(F) nfc(F) pppoe(F) pppox(F) ppp_generic(F) slhc(F) rds(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) ixgbe(F) iTCO_wdt(F) mdio(F) iTCO_vendor_support(F) dca(F) e1000e(F) lpc_ich(F) serio_raw(F) ptp(F) mfd_core(F) hpilo(F) pcspkr(F) hpwdt(F) pps_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) crc_t10dif(F) ata_generic(F) mgag200(F) i2c_algo _bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [10761.921495] CPU 2 [10761.930503] Pid: 63, comm: khugepaged Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 [10761.975282] RIP: 0010:[<ffffffff8118b7fa>] [<ffffffff8118b7fa>] hugepage_vma_check+0xa/0x70 [10762.020574] RSP: 0000:ffff8801fceedd60 EFLAGS: 00010206 [10762.049366] RAX: 0000000000000003 RBX: 0000000002e00000 RCX: ffff8801fceedfd8 [10762.087874] RDX: 0000000000000000 RSI: 0000000002f08000 RDI: 0000be0300000001 [10762.127078] RBP: ffff8801fceedd68 R08: ffff8801e0a15c00 R09: 0000000000000000 [10762.162283] R10: 0000000000000000 R11: ffff8801d9101000 R12: 0000000003000000 [10762.197992] R13: 0000be0300000001 R14: ffff8801d9101000 R15: 0000000000000202 [10762.234176] FS: 0000000000000000(0000) GS:ffff88020b440000(0000) knlGS:0000000000000000 [10762.274068] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10762.302770] CR2: 0000000000000000 CR3: 00000000018fa000 CR4: 00000000000407e0 [10762.338146] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10762.373845] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [10762.409883] Process khugepaged (pid: 63, threadinfo ffff8801fceec000, task ffff8801fceb3580) [10762.452060] Stack: [10762.462025] 0000000002e00000 ffff8801fceede40 ffffffff8118bb0c ffff8801fceedd90 [10762.498485] ffffffff810712d2 ffff8801ff640000 ffff8801fceede40 ffff8801f9a28a10 [10762.536010] ffff880000000000 ffff8801e56f28e8 ffff8801fceede70 ffffea0007644000 [10762.575870] Call Trace: [10762.589029] [<ffffffff8118bb0c>] khugepaged_scan_mm_slot+0xcc/0xd80 [10762.623447] [<ffffffff810712d2>] ? del_timer_sync+0x52/0x60 [10762.654387] [<ffffffff8118c8e3>] khugepaged+0x123/0x600 [10762.682370] [<ffffffff810850e0>] ? wake_up_bit+0x30/0x30 [10762.709898] [<ffffffff8118c7c0>] ? khugepaged_scan_mm_slot+0xd80/0xd80 [10762.742505] [<ffffffff81084240>] kthread+0xc0/0xd0 [10762.767903] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10762.799013] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [10762.826476] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [10762.856488] Code: 89 c4 e8 2a 25 48 00 49 63 c4 45 85 e4 48 0f 45 d8 48 89 d8 5b 41 5c 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 53 <48> 8b 47 50 48 89 fb a9 00 00 00 20 75 09 f6 05 b1 04 87 00 01 [10762.951263] RIP [<ffffffff8118b7fa>] hugepage_vma_check+0xa/0x70 [10762.982007] RSP <ffff8801fceedd60> [10762.999155] ---[ end trace aa0156eab3150a04 ]--- [-- MARK -- Thu May 30 07:45:00 2013] [10779.411271] hpsa 0000:04:00.0: Abort request on C0:B0:T0:L0 [10783.500306] INFO: rcu_sched detected stalls on CPUs/tasks: {} (detected by 3, t=60031 jiffies, g=677612, c=677611, q=125) [10783.559039] INFO: Stall ended before state dump start [10801.881812] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 1 [10802.953682] Shutting down cpus with NMI [10802.974055] drm_kms_helper: panic occurred, switching back to text console ----- Original Message ----- > From: "CAI Qian" <caiqian@redhat.com> > To: "Dave Chinner" <david@fromorbit.com> > Cc: xfs@oss.sgi.com, stable@vger.kernel.org > Sent: Thursday, May 30, 2013 1:40:32 PM > Subject: Re: 3.9.3: Oops running xfstests > > > > ----- Original Message ----- > > From: "Dave Chinner" <david@fromorbit.com> > > To: "CAI Qian" <caiqian@redhat.com> > > Cc: xfs@oss.sgi.com, stable@vger.kernel.org > > Sent: Thursday, May 30, 2013 1:20:49 PM > > Subject: Re: 3.9.3: Oops running xfstests > > > > On Wed, May 29, 2013 at 11:45:28PM -0400, CAI Qian wrote: > > > Also reproduced this on 3.9.4 on several different machines. > > > > How? what workload are you running that causes it to occur? > I have been running the following workloads, trinity, ltp (including > oom tests and some filesystem tests), some hugetlbfs tests, selinux > test suite, tracepoint tests, connectathon tests and then xfstests. > I am still trying to narrow down the minimal workload that could > reproduce this. > > > > <snip stack trace showing page overrun in > > xfs_attr_leaf_compact/xfs_attr_leaf_moveents> > > > > > System information is below: > > > > > > + cat /proc/mounts > > .... > > > /dev/mapper/rhel_hp--xw9400--02-root / xfs > > > rw,seclabel,relatime,attr2,inode64,noquota 0 0 > > > /dev/mapper/rhel_hp--xw9400--02-home /home xfs > > > rw,seclabel,relatime,attr2,inode64,noquota 0 0 > > > /dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 > > > > Which of these filesystems did it happen on? Or some other > > filesystem not mounted when you took this snapshot? > + export TEST_DEV=/dev/mapper/rhel_hp--bl495cg5--02-swap > + TEST_DEV=/dev/mapper/rhel_hp--bl495cg5--02-swap > + export TEST_DIR=/mnt/testarea/test > + TEST_DIR=/mnt/testarea/test > + export SCRATCH_DEV=/dev/mapper/rhel_hp--bl495cg5--02-home > + SCRATCH_DEV=/dev/mapper/rhel_hp--bl495cg5--02-home > + export SCRATCH_MNT=/mnt/testarea/scratch > + SCRATCH_MNT=/mnt/testarea/scratch > + mkdir -p /mnt/testarea/test > home = /dev/mapper/rhel_hp--bl495cg5--02-home > + mkdir -p /mnt/testarea/scratch > + mkfs.xfs -f /dev/mapper/rhel_hp--bl495cg5--02-swap > meta-data=/dev/mapper/rhel_hp--bl495cg5--02-swap isize=256 agcount=4, > agsize=510976 blks > = sectsz=512 attr=2, projid32bit=0 > data = bsize=4096 blocks=2043904, imaxpct=25 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 > log =internal log bsize=4096 blocks=2560, version=2 > = sectsz=512 sunit=0 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > + mkfs.xfs -f /dev/mapper/rhel_hp--bl495cg5--02-home > meta-data=/dev/mapper/rhel_hp--bl495cg5--02-home isize=256 agcount=4, > agsize=1103872 blks > = sectsz=512 attr=2, projid32bit=0 > data = bsize=4096 blocks=4415488, imaxpct=25 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 > log =internal log bsize=4096 blocks=2560, version=2 > = sectsz=512 sunit=0 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > > > > And can you post the xfs_info output of the filesystems under test, > > too? > xfs_info: /dev/mapper/rhel_hp--bl495cg5--02-swap is not a mounted XFS > filesystem > + echo > > + xfs_info /dev/mapper/rhel_hp--bl495cg5--02-home > meta-data=/dev/mapper/rhel_hp--bl495cg5--02-home isize=256 agcount=4, > agsize=1103872 blks > = sectsz=512 attr=2 > data = bsize=4096 blocks=4415488, imaxpct=25 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 > log =internal bsize=4096 blocks=2560, version=2 > = sectsz=512 sunit=0 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > CAI Qian > > > > Cheers, > > > > Dave. > > -- > > Dave Chinner > > david@fromorbit.com > > > _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-05-31 3:03 ` 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) CAI Qian @ 2013-05-31 5:35 ` Eric Sandeen 2013-05-31 6:04 ` Dave Chinner 1 sibling, 0 replies; 18+ messages in thread From: Eric Sandeen @ 2013-05-31 5:35 UTC (permalink / raw) To: CAI Qian; +Cc: linux-mm, LKML, stable, xfs On 5/30/13 10:03 PM, CAI Qian wrote: > OK, so the minimal workload to trigger this I found so far was to > run trinity, ltp and then xfstests. I have been able to easily > reproduced on 3 servers so far, and I'll post full logs here for > LKML and linux-mm as this may unrelated to XFS only. As far as > I can tell from the previous testing results, this has never been > reproduced back in 3.9 GA time. This seems also been reproduced > on 3.10-rc3 mostly on s390x so far. > CAI Qian > Can you hit it w/o trinity? I ask because trinity's stated goal is to fuzz and corrupt, right - so it's quite possible that blowing up later in xfs is a side effect? -Eric _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-05-31 3:03 ` 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) CAI Qian 2013-05-31 5:35 ` Eric Sandeen @ 2013-05-31 6:04 ` Dave Chinner 2013-06-03 3:04 ` CAI Qian 1 sibling, 1 reply; 18+ messages in thread From: Dave Chinner @ 2013-05-31 6:04 UTC (permalink / raw) To: CAI Qian; +Cc: linux-mm, LKML, stable, xfs On Thu, May 30, 2013 at 11:03:35PM -0400, CAI Qian wrote: > OK, so the minimal workload to trigger this I found so far was to > run trinity, ltp and then xfstests. I have been able to easily > reproduced on 3 servers so far, and I'll post full logs here for > LKML and linux-mm as this may unrelated to XFS only. As far as > I can tell from the previous testing results, this has never been > reproduced back in 3.9 GA time. This seems also been reproduced > on 3.10-rc3 mostly on s390x so far. > CAI Qian > > 1) > [10948.104344] XFS (dm-2): Mounting Filesystem > [10948.257335] XFS (dm-2): Ending clean mount > [10949.753951] XFS (dm-0): Mounting Filesystem > [10950.012824] XFS (dm-0): Ending clean mount > [10951.115722] BUG: unable to handle kernel paging request at ffff88003a9ca478 > [10951.122727] IP: [<ffffffff810f7f47>] __call_rcu.constprop.47+0x77/0x230 Note the timestamp there -I'll come back to it. This has blown up in rcu processing on a idle CPU.... > [10951.527453] BUG: unable to handle kernel paging request at ffff88003db52000 > [10951.527459] IP: [<ffffffff81302301>] memmove+0x51/0x1a0 .. aproximately 400ms before XFS has tripped over in xfs_attr_leaf_compact(). And for the second crash: > 2) > [20914.575680] XFS (dm-2): Mounting Filesystem > [20914.653966] XFS (dm-2): Ending clean mount > [20915.812263] XFS (dm-0): Mounting Filesystem > [20916.628209] XFS (dm-0): Ending clean mount > [20917.997069] divide error: 0000 [#1] SMP A divide by zero error in the scheduler find_busiest_group() function during CPU idle work, with no XFS traces at all. > 3) > [10721.161888] XFS (dm-0): Ending clean mount > [10722.090855] XFS (dm-2): Mounting Filesystem > [10722.143682] XFS (dm-2): Ending clean mount > [10722.584327] XFS (dm-0): Mounting Filesystem > [10722.949519] XFS (dm-0): Ending clean mount > [10723.579704] BUG: unable to handle kernel paging request at ffff8801b55cc000 > [10723.619185] IP: [<ffffffff81302305>] memmove+0x55/0x1a0 XFS trips over, almost simultaneously with: > [10724.128760] WARNING: at mm/mmap.c:2711 exit_mmap+0x166/0x170() A warning in exit_mmap() in a singal handler.. > [10724.128901] BUG: Bad rss-counter state mm:ffff8801fb90af40 idx:0 val:5 > [10724.128901] BUG: Bad rss-counter state mm:ffff8801fb90af40 idx:1 val:3 A bunch of obviously bad mm state warnings, and > [10724.128927] rhts-test-runne[48621]: segfault at 19cc ip 00000000004401c4 sp 00007fff27f840e0 error 6 in bash[400000+db000] > [10724.129726] ============================================================================= > [10724.129727] BUG kmalloc-4096 (Tainted: GF W ): Padding overwritten. 0xffff88019152fc28-0xffff88019152ffff > [10724.129727] ----------------------------------------------------------------------------- > [10724.129727] > [10724.129728] INFO: Slab 0xffffea0006454a00 objects=7 used=7 fp=0x (null) flags=0x20000000004080 > [10724.129729] Pid: 43729, comm: kworker/0:0 Tainted: GF B W 3.9.4 #1 > [10724.129729] Call Trace: > [10724.129730] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 > [10724.129733] [<ffffffff812f8007>] ? kset_init+0x27/0x40 > [10724.129735] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 > [10724.129736] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 > [10724.129739] [<ffffffff811820e3>] check_slab+0x73/0x100 > [10724.129740] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 > [10724.129742] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 > [10724.129744] [<ffffffff8109f5d8>] ? load_balance+0x108/0x7d0 > [10724.129746] [<ffffffff810ce4d2>] ? cgroup_release_agent+0x42/0x180 > [10724.129747] [<ffffffff8108fa7c>] ? update_rq_clock.part.67+0x1c/0x170 > [10724.129749] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 > [10724.129751] [<ffffffff810ce4d2>] cgroup_release_agent+0x42/0x180 cgroup slab memory poison overwrite warning. > [10724.135254] systemd[1]: segfault at 45 ip 00007f93e348c780 sp 00007fff4deb3a9e error 4 in libdbus-1.so.3.7.2[7f93e3479000+45000] > [10724.135272] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb34f8 error:0 in systemd[7f93e496d000+f6000] > [10724.135281] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb2f38 error:0 in systemd[7f93e496d000+f6000] > [10724.135289] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb2978 error:0 in systemd[7f93e496d000+f6000] > [10724.135297] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb23b8 error:0 in systemd[7f93e496d000+f6000] > [10724.135304] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb1df8 error:0 in systemd[7f93e496d000+f6000] > [10724.135312] traps: systemd[1] trap invalid opcode ip:7f93e498c490 sp:7fff4deb1838 error:0 in systemd[7f93e496d000+f6000] A bunch of nasty systemd trap warnings for illegal operations. FWIW: > [10748.689674] general protection fault: 0000 [#7] SMP > [10749.217082] CPU 2 > [10749.227162] Pid: 464, comm: flush-253:1 Tainted: GF B D W 3.9.4 #1 HP ProLiant DL120 G7 ... > [10749.882823] Call Trace: > [10749.895832] [<ffffffff81157aa2>] vma_interval_tree_iter_first+0x22/0x30 > [10749.932506] [<ffffffff8116714e>] page_mkclean+0x6e/0x1d0 > [10749.960173] [<ffffffff8113dbb8>] clear_page_dirty_for_io+0x48/0xc0 > [10749.991604] [<ffffffff8113de4a>] write_cache_pages+0x21a/0x4b0 > [10750.020918] [<ffffffff8113d4b0>] ? global_dirtyable_memory+0x50/0x50 > [10750.053649] [<ffffffff8113e120>] generic_writepages+0x40/0x60 > [10750.083106] [<ffffffffa01449e5>] xfs_vm_writepages+0x35/0x40 [xfs] > [10750.113998] [<ffffffff8113f02e>] do_writepages+0x1e/0x40 > [10750.141300] [<ffffffff811c2ec0>] __writeback_single_inode+0x40/0x210 > [10750.173209] [<ffffffff811c4b77>] writeback_sb_inodes+0x197/0x400 > [10750.204613] [<ffffffff811c4e7f>] __writeback_inodes_wb+0x9f/0xd0 > [10750.235080] [<ffffffff811c5543>] wb_writeback+0x233/0x2b0 > [10750.263128] [<ffffffff811c6f65>] wb_do_writeback+0x1e5/0x1f0 > [10750.291413] [<ffffffff811c6ffb>] bdi_writeback_thread+0x8b/0x210 > [10750.321825] [<ffffffff811c6f70>] ? wb_do_writeback+0x1f0/0x1f0 There's a clear indication that VM system is completely screwed up - The flusher thread crashed trying to clear the dirty page bit during data writeback. There's memory corruption all over the place. It is most likely that trinity is causing this - it's purpose is to trigger corruption issues, but they aren't always immediately seen. If you can trigger this xfs trace without trinity having been run and without all the RCU/idle/scheduler/cgroup issues occuring at the same time, then it's likely to be caused by XFS. But right now, I'd say XFS is just an innocent bystander caught in the crossfire. There's nothing I can do from an XFS persepctive to track this down... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-05-31 6:04 ` Dave Chinner @ 2013-06-03 3:04 ` CAI Qian 2013-06-03 4:00 ` Dave Chinner 0 siblings, 1 reply; 18+ messages in thread From: CAI Qian @ 2013-06-03 3:04 UTC (permalink / raw) To: Dave Chinner; +Cc: linux-mm, LKML, stable, xfs > There's memory corruption all over the place. It is most likely > that trinity is causing this - it's purpose is to trigger corruption > issues, but they aren't always immediately seen. If you can trigger > this xfs trace without trinity having been run and without all the > RCU/idle/scheduler/cgroup issues occuring at the same time, then > it's likely to be caused by XFS. But right now, I'd say XFS is just > an innocent bystander caught in the crossfire. There's nothing I can > do from an XFS persepctive to track this down... OK, this can be reproduced by just running LTP and then xfstests without trinity at all... [ 302.311213] XFS (dm-0): Mounting Filesystem [ 302.608320] XFS (dm-0): Ending clean mount [ 303.625760] XFS (dm-2): Mounting Filesystem [ 303.674648] XFS (dm-2): Ending clean mount [ 304.247740] XFS (dm-0): Mounting Filesystem [ 304.563899] XFS (dm-0): Ending clean mount [ 305.118268] BUG: unable to handle kernel paging request at ffff8801f7067000 [ 305.156637] IP: [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 305.185560] PGD 1ddf067 PUD 20bdf9067 PMD 1f9b56063 PTE 80000001f7067161 [ 305.222852] Oops: 0003 [#1] SMP [ 305.238742] Modules linked in: lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) iTCO_wdt(F) e1000e(F) ixgbe(F) iTCO_vendor_support(F) ptp(F) mdio(F) dca(F) serio_raw(F) hpwdt(F) pcspkr(F) pps_core(F) hpilo(F) lpc_ich(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) mgag200(F) ata_generic(F) crc_t10dif(F) i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 305.615528] CPU 3 [ 305.624597] Pid: 19138, comm: attr Tainted: GF 3.9.4 #1 HP ProLiant DL120 G7 [ 305.667290] RIP: 0010:[<ffffffff813022fa>] [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 305.707799] RSP: 0018:ffff8801da2a7ad8 EFLAGS: 00010282 [ 305.736413] RAX: ffff8801f7034540 RBX: ffff8801d3bba000 RCX: 000000000000d00b [ 305.774576] RDX: fffffffffffcd4f8 RSI: ffff8801f70673f8 RDI: ffff8801f7067000 [ 305.811143] RBP: ffff8801da2a7b30 R08: 3ee6e8057eed854d R09: 9066aaebe17ba23c [ 305.847038] R10: 0d8b3a7401f8834f R11: 74c08510ffdf89f6 R12: ffff8801f7034520 [ 305.883189] R13: 00000000000003d8 R14: 0000000000001800 R15: 000000000000007b [ 305.921160] FS: 00007fbdabf4e740(0000) GS:ffff88020b460000(0000) knlGS:0000000000000000 [ 305.961370] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 305.990403] CR2: ffff8801f7067000 CR3: 00000001d3d90000 CR4: 00000000000407e0 [ 306.025585] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 306.061610] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 306.097768] Process attr (pid: 19138, threadinfo ffff8801da2a6000, task ffff8801dfdb3580) [ 306.138073] Stack: [ 306.148523] ffffffffa01691a1 0000000000000000 0000000000000000 ffff8801f7034540 [ 306.187280] ffff8801f7034520 0000000000000018 ffff8801f7034520 0000000000000000 [ 306.227244] ffff8801e2425cd0 ffff8801d207e880 ffff8801d3e6ad20 ffff8801da2a7b68 [ 306.267164] Call Trace: [ 306.280421] [<ffffffffa01691a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [ 306.321077] [<ffffffffa0169467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [ 306.355110] [<ffffffffa016aa2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [ 306.386559] [<ffffffffa0166850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [ 306.420459] [<ffffffffa0174d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [ 306.453034] [<ffffffffa016778c>] xfs_attr_set_int+0x30c/0x420 [xfs] [ 306.484498] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [ 306.511064] [<ffffffffa0167d1f>] xfs_attr_set+0x7f/0x90 [xfs] [ 306.539972] [<ffffffffa015da12>] xfs_xattr_set+0x42/0x70 [xfs] [ 306.570403] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [ 306.598333] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [ 306.629324] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [ 306.655470] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [ 306.681365] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [ 306.710684] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [ 306.740728] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [ 306.767873] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [ 306.799781] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [ 306.831920] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [ 306.861316] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 306.894531] Code: 00 00 48 81 fa a8 02 00 00 72 05 40 38 fe 74 41 48 83 ea 20 48 83 ea 20 4c 8b 1e 4c 8b 56 08 4c 8b 4e 10 4c 8b 46 18 48 8d 76 20 <4c> 89 1f 4c 89 57 08 4c 89 4f 10 4c 89 47 18 48 8d 7f 20 73 d4 [ 306.989301] RIP [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 307.016021] RSP <ffff8801da2a7ad8> [ 307.033212] CR2: ffff8801f7067000 [ 307.049503] ---[ end trace 94a32a2cfce117d8 ]--- [ 307.073111] BUG: unable to handle kernel paging request at 0000428200000008 [ 307.107376] IP: [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [ 307.135985] PGD 0 [ 307.145833] Oops: 0000 [#2] SMP [ 307.161695] Modules linked in: lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) iTCO_wdt(F) e1000e(F) ixgbe(F) iTCO_vendor_support(F) ptp(F) mdio(F) dca(F) serio_raw(F) hpwdt(F) pcspkr(F) pps_core(F) hpilo(F) lpc_ich(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) mgag200(F) ata_generic(F) crc_t10dif(F) i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 307.547533] CPU 3 [ 307.556537] Pid: 18958, comm: sleep Tainted: GF D 3.9.4 #1 HP ProLiant DL120 G7 [ 307.598058] RIP: 0010:[<ffffffff810c8b00>] [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [ 307.638608] RSP: 0018:ffff8801fd1e3e90 EFLAGS: 00010206 [ 307.664796] RAX: 0000428200000000 RBX: ffff8801ff0a5d80 RCX: ffff8801fd1e3fd8 [ 307.700780] RDX: 0000000006936000 RSI: 0000000000000001 RDI: ffff8801d3aef328 [ 307.735921] RBP: ffff8801fd1e3eb0 R08: ffff8801dff5d040 R09: 0000000000000000 [ 307.771698] R10: ffffffffffffff80 R11: 0000000000000246 R12: ffff8801dff5d040 [ 307.808645] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8801dff5d040 [ 307.847144] FS: 00007f3a74a7b740(0000) GS:ffff88020b460000(0000) knlGS:0000000000000000 [ 307.891651] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 307.922666] CR2: 0000428200000008 CR3: 00000001d21f7000 CR4: 00000000000407e0 [ 307.961110] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 307.999904] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 308.035206] Process sleep (pid: 18958, threadinfo ffff8801fd1e2000, task ffff8801dff5d040) [ 308.076654] Stack: [ 308.086497] 0000000000000000 0000000000004a0e ffff8801dff5d040 00007fffa372b608 [ 308.123532] ffff8801fd1e3f38 ffffffff81066e01 0000000100000000 0000000000000000 [ 308.160362] 00000046b1319ffe 00000046b130dcae ffffffff81087820 ffff88020b46e700 [ 308.197286] Call Trace: [ 308.209307] [<ffffffff81066e01>] do_exit+0x7e1/0xa20 [ 308.234897] [<ffffffff81087820>] ? hrtimer_get_res+0x50/0x50 [ 308.263462] [<ffffffff8160e341>] ? do_nanosleep+0x61/0xe0 [ 308.291687] [<ffffffff810670bf>] do_group_exit+0x3f/0xa0 [ 308.318371] [<ffffffff81067134>] sys_exit_group+0x14/0x20 [ 308.346294] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 308.376928] Code: 00 00 00 74 56 49 8b 84 24 80 04 00 00 48 8d 78 68 e8 25 5b 54 00 49 8b 84 24 80 04 00 00 48 8b 00 48 85 c0 74 1c 31 d2 0f 1f 00 <48> 03 50 08 48 2b 10 48 8b 40 10 48 85 c0 75 f0 48 c1 ea 0a 49 [ 308.480424] RIP [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [ 308.509509] RSP <ffff8801fd1e3e90> [ 308.527362] CR2: 0000428200000008 [ 308.543697] general protection fault: 0000 [#3] [ 308.543760] ---[ end trace 94a32a2cfce117d9 ]--- [ 308.543761] Fixing recursive fault but reboot is needed! SMP [ 308.618050] Modules linked in: lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) iTCO_wdt(F) e1000e(F) ixgbe(F) iTCO_vendor_support(F) ptp(F) mdio(F) dca(F) serio_raw(F) hpwdt(F) pcspkr(F) pps_core(F) hpilo(F) lpc_ich(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) mgag200(F) ata_generic(F) crc_t10dif(F) i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 309.002756] CPU 0 [ 309.012338] Pid: 19138, comm: attr Tainted: GF D 3.9.4 #1 HP ProLiant DL120 G7 [ 309.056022] RIP: 0010:[<ffffffff8115798b>] [<ffffffff8115798b>] vma_interval_tree_remove+0x14b/0x240 [ 309.101815] RSP: 0018:ffff8801da2a7670 EFLAGS: 00010202 [ 309.128508] RAX: ffff8801f703d058 RBX: ffff8801fb9f0e00 RCX: 6b6b6b6b6b6b6b6b [ 309.163784] RDX: 6b6b6b6b6b6b6b6a RSI: ffff8801febdb220 RDI: ffff8801fb9f0e00 [ 309.199476] RBP: ffff8801da2a7688 R08: 0000000000000000 R09: 0000000000017700 [ 309.235490] R10: ffff8801fb9f0e58 R11: 000000000000000e R12: ffff8801fb9f0e00 [ 309.270754] R13: ffff8801febdb238 R14: ffff8801febdb200 R15: ffff8801fb9f2a00 [ 309.307101] FS: 00007fbdabf4e740(0000) GS:ffff88020b400000(0000) knlGS:0000000000000000 [ 309.348288] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 309.376730] CR2: 0000000002077840 CR3: 00000001d3d90000 CR4: 00000000000407f0 [ 309.413051] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 309.449081] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 309.487646] Process attr (pid: 19138, threadinfo ffff8801da2a6000, task ffff8801dfdb3580) [ 309.532131] Stack: [ 309.543173] ffffffff811613e6 ffff8801d21718c0 ffff8801fb9f0e00 ffff8801da2a76b8 [ 309.582822] ffffffff811619a0 ffff8801fb9f0e00 00007fbdab776000 0000000000000000 [ 309.621914] ffff8801da2a7710 ffff8801da2a76f8 ffffffff8115a1d8 0000000000000000 [ 309.659092] Call Trace: [ 309.671075] [<ffffffff811613e6>] ? __remove_shared_vm_struct+0x46/0x70 [ 309.704907] [<ffffffff811619a0>] unlink_file_vma+0x40/0x60 [ 309.732536] [<ffffffff8115a1d8>] free_pgtables+0x48/0x120 [ 309.760022] [<ffffffff81164722>] exit_mmap+0xb2/0x170 [ 309.785585] [<ffffffff8160eda5>] ? __schedule+0x3b5/0x7d0 [ 309.813041] [<ffffffff8105e377>] mmput+0x67/0xf0 [ 309.837444] [<ffffffff81066898>] do_exit+0x278/0xa20 [ 309.862503] [<ffffffff81604989>] ? printk+0x67/0x69 [ 309.888016] [<ffffffff81063088>] ? console_unlock+0x1e8/0x3f0 [ 309.916848] [<ffffffff81612034>] oops_end+0xa4/0xe0 [ 309.942105] [<ffffffff81604212>] no_context+0x263/0x270 [ 309.968280] [<ffffffff81604292>] __bad_area_nosemaphore+0x73/0x1cc [ 310.000888] [<ffffffff816043fe>] bad_area_nosemaphore+0x13/0x15 [ 310.033080] [<ffffffff81614c0e>] __do_page_fault+0x31e/0x510 [ 310.064304] [<ffffffffa0161497>] ? kmem_alloc+0x67/0xf0 [xfs] [ 310.095564] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [ 310.125723] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [ 310.153458] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [ 310.182482] [<ffffffff81614e0e>] do_page_fault+0xe/0x10 [ 310.209285] [<ffffffff81611498>] page_fault+0x28/0x30 [ 310.234674] [<ffffffff813022fa>] ? memmove+0x4a/0x1a0 [ 310.260888] [<ffffffffa01691a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [ 310.298854] [<ffffffffa0169467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [ 310.332735] [<ffffffffa016aa2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [ 310.364814] [<ffffffffa0166850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [ 310.398089] [<ffffffffa0174d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [ 310.431514] [<ffffffffa016778c>] xfs_attr_set_int+0x30c/0x420 [xfs] [ 310.462923] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [ 310.489423] [<ffffffffa0167d1f>] xfs_attr_set+0x7f/0x90 [xfs] [ 310.518114] [<ffffffffa015da12>] xfs_xattr_set+0x42/0x70 [xfs] [ 310.548571] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [ 310.579571] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [ 310.613506] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [ 310.640943] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [ 310.668401] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [ 310.700284] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [ 310.727708] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [ 310.753288] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [ 310.782891] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [ 310.812647] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [ 310.839684] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 310.870818] Code: 48 89 50 18 48 8b 00 48 83 e0 fc 74 43 48 8b 50 b0 48 2b 50 a8 48 8b 48 40 48 c1 ea 0c 48 8d 54 11 ff 48 8b 48 10 48 85 c9 74 0b <48> 8b 49 18 48 39 ca 48 0f 42 d1 48 8b 48 08 48 85 c9 74 0b 48 [ 310.966164] RIP [<ffffffff8115798b>] vma_interval_tree_remove+0x14b/0x240 [ 311.000221] RSP <ffff8801da2a7670> [ 311.017819] BUG: unable to handle kernel [ 311.017973] ---[ end trace 94a32a2cfce117da ]--- [ 311.017974] Fixing recursive fault but reboot is needed! paging request at 00007f7f3985cfe8 [ 311.105330] IP: [<ffffffff81160ff3>] find_vma+0x43/0xa0 [ 311.133081] PGD 0 [ 311.144699] Oops: 0000 [#4] SMP [ 311.161948] Modules linked in: lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) iTCO_wdt(F) e1000e(F) ixgbe(F) iTCO_vendor_support(F) ptp(F) mdio(F) dca(F) serio_raw(F) hpwdt(F) pcspkr(F) pps_core(F) hpilo(F) lpc_ich(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) mgag200(F) ata_generic(F) crc_t10dif(F) i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 311.542611] CPU 1 [ 311.552199] Pid: 17036, comm: systemd-udevd Tainted: GF D 3.9.4 #1 HP ProLiant DL120 G7 [ 311.596657] RIP: 0010:[<ffffffff81160ff3>] [<ffffffff81160ff3>] find_vma+0x43/0xa0 [ 311.636364] RSP: 0000:ffff8801d16ede48 EFLAGS: 00010206 [ 311.665124] RAX: 0000000000000000 RBX: 0000000000000014 RCX: ffff8801d16edfd8 [ 311.703530] RDX: 00007f7f3985d000 RSI: 00007fb5f3715a90 RDI: ffff8801d3aee540 [ 311.741784] RBP: ffff8801d16edf38 R08: 0000000000000001 R09: 00007fb5f28317d8 [ 311.777051] R10: 00007fb5f28317d8 R11: 0000000000000246 R12: 0000000000000028 [ 311.813094] R13: 00007fb5f3715a90 R14: ffff8801d16edf58 R15: ffff8801d3aee540 [ 311.848908] FS: 00007fb5f36e0880(0000) GS:ffff88020b420000(0000) knlGS:0000000000000000 [ 311.890296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 311.919321] CR2: 00007f7f3985cfe8 CR3: 00000001fbef8000 CR4: 00000000000407e0 [ 311.954562] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 311.990646] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 312.026415] Process systemd-udevd (pid: 17036, threadinfo ffff8801d16ec000, task ffff8801f70d8000) [ 312.071166] Stack: [ 312.081624] ffffffff816149f3 ffff8801d12620dc ffff8801f70d8000 ffff8801d3aee5a8 [ 312.119621] ffff8801d16ede90 ffffffff811b9a9e ffff8801d16418c0 0000000000000008 [ 312.159495] ffff8801feb0d340 ffff8801d16edea0 ffffffff811b9ba6 ffff8801d16edee8 [ 312.199195] Call Trace: [ 312.212470] [<ffffffff816149f3>] ? __do_page_fault+0x103/0x510 [ 312.244953] [<ffffffff811b9a9e>] ? mntput_no_expire+0x3e/0x120 [ 312.277378] [<ffffffff811b9ba6>] ? mntput+0x26/0x40 [ 312.302467] [<ffffffff8119d9e3>] ? __fput+0x183/0x270 [ 312.327922] [<ffffffff8119db8e>] ? ____fput+0xe/0x10 [ 312.353480] [<ffffffff81080f2c>] ? task_work_run+0xac/0xe0 [ 312.381167] [<ffffffff81614e0e>] do_page_fault+0xe/0x10 [ 312.408212] [<ffffffff81611498>] page_fault+0x28/0x30 [ 312.433289] Code: 47 48 8b 57 08 31 c0 48 85 d2 75 1f eb 44 66 2e 0f 1f 84 00 00 00 00 00 48 3b 72 e0 48 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f <48> 3b 72 e8 72 e7 48 8b 52 08 48 85 d2 75 f1 48 85 c0 74 10 48 [ 312.528198] RIP [<ffffffff81160ff3>] find_vma+0x43/0xa0 [ 312.555203] RSP <ffff8801d16ede48> [ 312.572391] CR2: 00007f7f3985cfe8 [ 312.588724] ---[ end trace 94a32a2cfce117db ]--- [ 312.612043] BUG: unable to handle kernel paging request at 0000428200000008 [ 312.646416] IP: [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [ 312.675123] PGD 0 [ 312.685662] Oops: 0000 [#5] SMP [ 312.702907] Modules linked in: lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) iTCO_wdt(F) e1000e(F) ixgbe(F) iTCO_vendor_support(F) ptp(F) mdio(F) dca(F) serio_raw(F) hpwdt(F) pcspkr(F) pps_core(F) hpilo(F) lpc_ich(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) mgag200(F) ata_generic(F) crc_t10dif(F) i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 313.090194] CPU 2 [ 313.099192] Pid: 17036, comm: systemd-udevd Tainted: GF D 3.9.4 #1 HP ProLiant DL120 G7 [ 313.143933] RIP: 0010:[<ffffffff810c8b00>] [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [ 313.183945] RSP: 0018:ffff8801d16edaf8 EFLAGS: 00010206 [ 313.210509] RAX: 0000428200000000 RBX: ffff8801dfefac00 RCX: ffff8801d16edfd8 [ 313.246545] RDX: 00000000029b7000 RSI: 0000000000000001 RDI: ffff8801d3aee5a8 [ 313.285230] RBP: ffff8801d16edb18 R08: ffff8801f70d8000 R09: 0000000000000000 [ 313.323961] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8801f70d8000 [ 313.362768] R13: 0000000000000009 R14: 0000000000000000 R15: ffff8801f70d8000 [ 313.402688] FS: 00007fb5f36e0880(0000) GS:ffff88020b440000(0000) knlGS:0000000000000000 [ 313.445042] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 313.474147] CR2: 0000428200000008 CR3: 00000001fbef8000 CR4: 00000000000407e0 [ 313.510209] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 313.545396] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 313.581255] Process systemd-udevd (pid: 17036, threadinfo ffff8801d16ec000, task ffff8801f70d8000) [ 313.626232] Stack: [ 313.636139] 0000000000000009 000000000000428c 0000000000000246 0000000000000000 [ 313.672927] ffff8801d16edba0 ffffffff81066e01 00000001d16edb80 ffffffff81604989 [ 313.709663] 0000000000000010 ffff8801d16edb90 ffff8801d16edb50 ffffffff819f6f88 [ 313.746821] Call Trace: [ 313.759043] [<ffffffff81066e01>] do_exit+0x7e1/0xa20 [ 313.784649] [<ffffffff81604989>] ? printk+0x67/0x69 [ 313.809568] [<ffffffff81612034>] oops_end+0xa4/0xe0 [ 313.834180] [<ffffffff81604212>] no_context+0x263/0x270 [ 313.863369] [<ffffffff81604292>] __bad_area_nosemaphore+0x73/0x1cc [ 313.896335] [<ffffffff8160460f>] bad_area+0x43/0x4a [ 313.924110] [<ffffffff81614d4c>] __do_page_fault+0x45c/0x510 [ 313.955197] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [ 313.986579] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [ 314.012728] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [ 314.039782] [<ffffffff81606e25>] ? free_debug_processing+0x1de/0x222 [ 314.071858] [<ffffffff81614e0e>] do_page_fault+0xe/0x10 [ 314.098809] [<ffffffff81611498>] page_fault+0x28/0x30 [ 314.125269] [<ffffffff81160ff3>] ? find_vma+0x43/0xa0 [ 314.152148] [<ffffffff816149f3>] ? __do_page_fault+0x103/0x510 [ 314.181402] [<ffffffff811b9a9e>] ? mntput_no_expire+0x3e/0x120 [ 314.211469] [<ffffffff811b9ba6>] ? mntput+0x26/0x40 [ 314.236258] [<ffffffff8119d9e3>] ? __fput+0x183/0x270 [ 314.262654] [<ffffffff8119db8e>] ? ____fput+0xe/0x10 [ 314.287521] [<ffffffff81080f2c>] ? task_work_run+0xac/0xe0 [ 314.315854] [<ffffffff81614e0e>] do_page_fault+0xe/0x10 [ 314.342280] [<ffffffff81611498>] page_fault+0x28/0x30 [ 314.368412] Code: 00 00 00 74 56 49 8b 84 24 80 04 00 00 48 8d 78 68 e8 25 5b 54 00 49 8b 84 24 80 04 00 00 48 8b 00 48 85 c0 74 1c 31 d2 0f 1f 00 <48> 03 50 08 48 2b 10 48 8b 40 10 48 85 c0 75 f0 48 c1 ea 0a 49 [ 314.472334] RIP [<ffffffff810c8b00>] acct_collect+0x60/0x1b0 [ 314.502845] RSP <ffff8801d16edaf8> [ 314.520964] CR2: 0000428200000008 [ 314.537469] ---[ end trace 94a32a2cfce117dc ]--- [ 314.560276] Fixing recursive fault but reboot is needed! [ 314.598307] ============================================================================= [ 314.639471] BUG kmalloc-4096 (Tainted: GF D ): Padding overwritten. 0xffff8801f7037c28-0xffff8801f7037fff [ 314.691639] ----------------------------------------------------------------------------- [ 314.691639] [ 314.739818] INFO: Slab 0xffffea0007dc0c00 objects=7 used=4 fp=0xffff8801f7031148 flags=0x20000000004081 [ 314.786319] Pid: 302, comm: kworker/0:1H Tainted: GF B D 3.9.4 #1 [ 314.820301] Call Trace: [ 314.832314] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 [ 314.857597] [<ffffffff810c0007>] ? on_each_cpu_mask+0x47/0x60 [ 314.886444] [<ffffffff810154cf>] ? dump_trace+0x16f/0x2b0 [ 314.915497] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 [ 314.949702] [<ffffffff811820e3>] check_slab+0x73/0x100 [ 314.978505] [<ffffffff81180007>] ? page_referenced_ksm+0x77/0x1a0 [ 315.012162] [<ffffffff81606c9e>] free_debug_processing+0x57/0x222 [ 315.045042] [<ffffffffa0161595>] ? kmem_free+0x35/0x40 [xfs] [ 315.076862] [<ffffffffa0161595>] ? kmem_free+0x35/0x40 [xfs] [ 315.105209] [<ffffffff81606ea3>] __slab_free+0x3a/0x318 [ 315.132235] [<ffffffffa0161595>] ? kmem_free+0x35/0x40 [xfs] [ 315.160550] [<ffffffffa014e532>] ? xfs_extent_busy_clear_one+0x42/0xd0 [xfs] [ 315.196283] [<ffffffffa0161595>] ? kmem_free+0x35/0x40 [xfs] [ 315.224568] [<ffffffff8118373f>] kfree+0xff/0x130 [ 315.248848] [<ffffffffa0161595>] kmem_free+0x35/0x40 [xfs] [ 315.276362] [<ffffffffa01a0adf>] xlog_cil_committed+0x9f/0xe0 [xfs] [ 315.308507] [<ffffffffa019def0>] xlog_state_do_callback+0x170/0x2c0 [xfs] [ 315.342483] [<ffffffffa019e0ae>] xlog_state_done_syncing+0x6e/0x90 [xfs] [ 315.376540] [<ffffffffa019e11b>] xlog_iodone+0x4b/0xa0 [xfs] [ 315.405695] [<ffffffffa014b866>] xfs_buf_iodone_work+0x46/0xd0 [xfs] [ 315.437442] [<ffffffff8107d80f>] process_one_work+0x16f/0x3f0 [ 315.467401] [<ffffffff8107f360>] worker_thread+0x110/0x3d0 [ 315.497398] [<ffffffff8107f250>] ? __alloc_workqueue_key+0x500/0x500 [ 315.532523] [<ffffffff81084240>] kthread+0xc0/0xd0 [ 315.559039] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 315.592030] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [ 315.621438] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 315.652053] Padding ffff8801f70378f8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 315.699076] Padding ffff8801f7037908: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 315.745979] Padding ffff8801f7037918: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 315.792851] Padding ffff8801f7037928: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 315.840050] Padding ffff8801f7037938: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 315.887178] Padding ffff8801f7037948: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 315.934795] Padding ffff8801f7037958: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 315.982248] Padding ffff8801f7037968: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.030041] Padding ffff8801f7037978: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.080829] Padding ffff8801f7037988: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.131932] Padding ffff8801f7037998: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.183194] Padding ffff8801f70379a8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.233916] Padding ffff8801f70379b8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.281215] Padding ffff8801f70379c8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.328966] Padding ffff8801f70379d8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.376138] Padding ffff8801f70379e8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.422472] Padding ffff8801f70379f8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.469480] Padding ffff8801f7037a08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.516904] Padding ffff8801f7037a18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.564165] Padding ffff8801f7037a28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.612961] Padding ffff8801f7037a38: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.664073] Padding ffff8801f7037a48: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.715212] Padding ffff8801f7037a58: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.764149] Padding ffff8801f7037a68: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.811182] Padding ffff8801f7037a78: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.858494] Padding ffff8801f7037a88: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.906132] Padding ffff8801f7037a98: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 316.954730] Padding ffff8801f7037aa8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.001916] Padding ffff8801f7037ab8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.049322] Padding ffff8801f7037ac8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.097121] Padding ffff8801f7037ad8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.145051] Padding ffff8801f7037ae8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.196764] Padding ffff8801f7037af8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.247078] Padding ffff8801f7037b08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.297532] Padding ffff8801f7037b18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.345002] Padding ffff8801f7037b28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.392045] Padding ffff8801f7037b38: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.439071] Padding ffff8801f7037b48: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.485932] Padding ffff8801f7037b58: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.532876] Padding ffff8801f7037b68: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.580171] Padding ffff8801f7037b78: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.627661] Padding ffff8801f7037b88: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.674527] Padding ffff8801f7037b98: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.725010] Padding ffff8801f7037ba8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.775448] Padding ffff8801f7037bb8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.826466] Padding ffff8801f7037bc8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.873530] Padding ffff8801f7037bd8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.920442] Padding ffff8801f7037be8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 317.968769] Padding ffff8801f7037bf8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 318.015333] Padding ffff8801f7037c08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 318.062740] Padding ffff8801f7037c18: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ [ 318.109415] Padding ffff8801f7037c28: e0 bd 03 f7 01 88 ff ff ff ff ff ff ff ff ff ff ................ [ 318.156333] Padding ffff8801f7037c38: ff ff ff ff ff ff ff ff 80 01 00 00 00 00 00 00 ................ [ 318.203365] Padding ffff8801f7037c48: cc cc cc cc cc cc cc cc 68 81 03 f7 01 88 ff ff ........h....... [ 318.252859] Padding ffff8801f7037c58: 27 1e 2a 81 ff ff ff ff 2f 77 60 81 ff ff ff ff '.*...../w`..... [ 318.303185] Padding ffff8801f7037c68: d1 4d 18 81 ff ff ff ff 27 1e 2a 81 ff ff ff ff .M......'.*..... [ 318.354505] Padding ffff8801f7037c78: 55 2c 2a 81 ff ff ff ff 0d 30 2a 81 ff ff ff ff U,*......0*..... [ 318.404617] Padding ffff8801f7037c88: 77 bf 2a 81 ff ff ff ff a3 07 2a 81 ff ff ff ff w.*.......*..... [ 318.451210] Padding ffff8801f7037c98: 08 ec 29 81 ff ff ff ff df c6 19 81 ff ff ff ff ..)............. [ 318.497880] Padding ffff8801f7037ca8: d9 c9 19 81 ff ff ff ff 59 93 61 81 ff ff ff ff ........Y.a..... [ 318.544873] Padding ffff8801f7037cb8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.592075] Padding ffff8801f7037cc8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.639232] Padding ffff8801f7037cd8: 00 00 00 00 00 00 00 00 00 00 00 00 d7 21 00 00 .............!.. [ 318.686150] Padding ffff8801f7037ce8: 14 b3 fc ff 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.733295] Padding ffff8801f7037cf8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.780243] Padding ffff8801f7037d08: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.829838] Padding ffff8801f7037d18: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.881128] Padding ffff8801f7037d28: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.931308] Padding ffff8801f7037d38: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 318.979077] Padding ffff8801f7037d48: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.026194] Padding ffff8801f7037d58: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.073630] Padding ffff8801f7037d68: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.120712] Padding ffff8801f7037d78: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.167629] Padding ffff8801f7037d88: 5a 5a 5a 5a 5a 5a 5a 5a 78 bc 03 f7 01 88 ff ff ZZZZZZZZx....... [ 319.214535] Padding ffff8801f7037d98: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ................ [ 319.261416] Padding ffff8801f7037da8: 80 02 00 00 00 00 00 00 cc cc cc cc cc cc cc cc ................ [ 319.307982] Padding ffff8801f7037db8: d0 82 03 f7 01 88 ff ff 27 1e 2a 81 ff ff ff ff ........'.*..... [ 319.358389] Padding ffff8801f7037dc8: 2f 77 60 81 ff ff ff ff d1 4d 18 81 ff ff ff ff /w`......M...... [ 319.409888] Padding ffff8801f7037dd8: 27 1e 2a 81 ff ff ff ff 55 2c 2a 81 ff ff ff ff '.*.....U,*..... [ 319.460846] Padding ffff8801f7037de8: 0d 30 2a 81 ff ff ff ff 77 bf 2a 81 ff ff ff ff .0*.....w.*..... [ 319.508176] Padding ffff8801f7037df8: a3 07 2a 81 ff ff ff ff 08 ec 29 81 ff ff ff ff ..*.......)..... [ 319.555362] Padding ffff8801f7037e08: df c6 19 81 ff ff ff ff d9 c9 19 81 ff ff ff ff ................ [ 319.602255] Padding ffff8801f7037e18: 59 93 61 81 ff ff ff ff 00 00 00 00 00 00 00 00 Y.a............. [ 319.649541] Padding ffff8801f7037e28: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.697595] Padding ffff8801f7037e38: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.744772] Padding ffff8801f7037e48: 00 00 00 00 d7 21 00 00 14 b3 fc ff 00 00 00 00 .....!.......... [ 319.791652] Padding ffff8801f7037e58: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.838095] Padding ffff8801f7037e68: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.888739] Padding ffff8801f7037e78: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.939962] Padding ffff8801f7037e88: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 319.991256] Padding ffff8801f7037e98: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.039515] Padding ffff8801f7037ea8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.086594] Padding ffff8801f7037eb8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.133629] Padding ffff8801f7037ec8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.180672] Padding ffff8801f7037ed8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.227713] Padding ffff8801f7037ee8: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ [ 320.274136] Padding ffff8801f7037ef8: 00 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ................ [ 320.321932] Padding ffff8801f7037f08: ff ff ff ff ff ff ff ff 80 03 00 00 00 00 00 00 ................ [ 320.370826] Padding ffff8801f7037f18: cc cc cc cc cc cc cc cc 38 84 03 f7 01 88 ff ff ........8....... [ 320.420767] Padding ffff8801f7037f28: 27 1e 2a 81 ff ff ff ff 2f 77 60 81 ff ff ff ff '.*...../w`..... [ 320.471678] Padding ffff8801f7037f38: d1 4d 18 81 ff ff ff ff 27 1e 2a 81 ff ff ff ff .M......'.*..... [ 320.522943] Padding ffff8801f7037f48: 55 2c 2a 81 ff ff ff ff 0d 30 2a 81 ff ff ff ff U,*......0*..... [ 320.571368] Padding ffff8801f7037f58: 77 bf 2a 81 ff ff ff ff a3 07 2a 81 ff ff ff ff w.*.......*..... [ 320.618584] Padding ffff8801f7037f68: 08 ec 29 81 ff ff ff ff df c6 19 81 ff ff ff ff ..)............. [ 320.665595] Padding ffff8801f7037f78: d9 c9 19 81 ff ff ff ff 59 93 61 81 ff ff ff ff ........Y.a..... [ 320.713267] Padding ffff8801f7037f88: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.759666] Padding ffff8801f7037f98: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.806540] Padding ffff8801f7037fa8: 00 00 00 00 00 00 00 00 00 00 00 00 d7 21 00 00 .............!.. [ 320.854175] Padding ffff8801f7037fb8: 14 b3 fc ff 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.901359] Padding ffff8801f7037fc8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 320.950409] Padding ffff8801f7037fd8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 321.001549] Padding ffff8801f7037fe8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 321.052274] Padding ffff8801f7037ff8: 00 00 00 00 00 00 00 00 ........ [ 321.099932] FIX kmalloc-4096: Restoring 0xffff8801f70378f8-0xffff8801f7037fff=0x5a [ 321.099932] [ 343.457847] general protection fault: 0000 [#6] SMP [ 343.485571] Modules linked in: lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) iTCO_wdt(F) e1000e(F) ixgbe(F) iTCO_vendor_support(F) ptp(F) mdio(F) dca(F) serio_raw(F) hpwdt(F) pcspkr(F) pps_core(F) hpilo(F) lpc_ich(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) mgag200(F) ata_generic(F) crc_t10dif(F) i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 343.867974] CPU 3 [ 343.876992] Pid: 1119, comm: master Tainted: GF B D 3.9.4 #1 HP ProLiant DL120 G7 [ 343.918848] RIP: 0010:[<ffffffff811e1cb5>] [<ffffffff811e1cb5>] sys_epoll_ctl+0x135/0x7c0 [ 343.960461] RSP: 0018:ffff8801e2f65ee8 EFLAGS: 00010206 [ 343.986626] RAX: ffff8801e262b580 RBX: ffff8801dff6b840 RCX: ffff8801e2f65fd8 [ 344.025028] RDX: 0000000000000000 RSI: ffffffff8196ff30 RDI: ffff8801e21ed350 [ 344.063326] RBP: ffff8801e2f65f78 R08: 5a5a5a5a5a5a5a5a R09: 0101010101010101 [ 344.103131] R10: 00007fffc305ba30 R11: 0000000000000297 R12: 0000000000000001 [ 344.140479] R13: 0000000000000015 R14: ffff8801d2177bc0 R15: ffff8801e21ed348 [ 344.175959] FS: 00007facb165d840(0000) GS:ffff88020b460000(0000) knlGS:0000000000000000 [ 344.217871] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 344.246145] CR2: 00007fbb90d88e70 CR3: 00000001e62c3000 CR4: 00000000000407e0 [ 344.282166] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 344.317848] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 344.353523] Process master (pid: 1119, threadinfo ffff8801e2f64000, task ffff8801e262b580) [ 344.395286] Stack: [ 344.405276] 0000000000000001 0000000000000002 ffff8801e18a0f30 00007facb1696821 [ 344.442554] ffff8801e21ed350 00000001e3390790 ffff8801e21ed348 00000001e3390790 [ 344.479873] 0000000100000015 ffff8801e3390790 ffff8801e3390790 ffff8801e2f65f78 [ 344.516638] Call Trace: [ 344.529475] [<ffffffff810e0ef6>] ? __audit_syscall_exit+0x1f6/0x2a0 [ 344.564393] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 344.597127] Code: f8 01 0f 86 45 01 00 00 4c 8b 7c 24 30 4c 89 f8 48 83 c0 08 48 89 c7 48 89 44 24 20 e8 75 c0 42 00 4d 8b 47 68 90 4d 85 c0 74 1d <4d> 3b 70 30 77 0e 72 63 44 89 e8 41 2b 40 38 83 f8 00 7e 4d 4d [ 344.700053] RIP [<ffffffff811e1cb5>] sys_epoll_ctl+0x135/0x7c0 [ 344.729172] RSP <ffff8801e2f65ee8> [ 344.747188] ---[ end trace 94a32a2cfce117dd ]--- [ 373.803073] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 1 [ 373.844067] drm_kms_helper: panic occurred, switching back to text console [ 373.882178] ------------[ cut here ]------------ [ 373.908207] WARNING: at arch/x86/kernel/smp.c:123 native_smp_send_reschedule+0x5a/0x60() [ 373.948563] Hardware name: ProLiant DL120 G7 [ 373.969969] Modules linked in: lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) iTCO_wdt(F) e1000e(F) ixgbe(F) iTCO_vendor_support(F) ptp(F) mdio(F) dca(F) serio_raw(F) hpwdt(F) pcspkr(F) pps_core(F) hpilo(F) lpc_ich(F) mfd_core(F) microcode(F) xfs(F) libcrc32c(F) sd_mod(F) mgag200(F) ata_generic(F) crc_t10dif(F) i2c_algo_bit(F) pata_acpi(F) drm_kms_helper(F) ttm(F) ata_piix(F) drm(F) hpsa(F) libata(F) i2c_core(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 374.346510] Pid: 8673, comm: ntpd Tainted: GF B D 3.9.4 #1 [ 374.379076] Call Trace: [ 374.392291] <IRQ> [<ffffffff81060f50>] warn_slowpath_common+0x70/0xa0 [ 374.428962] [<ffffffff8106103a>] warn_slowpath_null+0x1a/0x20 [ 374.460338] [<ffffffff8103697a>] native_smp_send_reschedule+0x5a/0x60 [ 374.496389] [<ffffffff810a0545>] trigger_load_balance+0x175/0x210 [ 374.527311] [<ffffffff810943af>] scheduler_tick+0xff/0x140 [ 374.555719] [<ffffffff81071f46>] update_process_times+0x66/0x80 [ 374.585366] [<ffffffff810ba1c5>] tick_sched_handle.isra.13+0x25/0x60 [ 374.617683] [<ffffffff810ba241>] tick_sched_timer+0x41/0x60 [ 374.645722] [<ffffffff81087af4>] __run_hrtimer+0x74/0x1d0 [ 374.673706] [<ffffffff810ba200>] ? tick_sched_handle.isra.13+0x60/0x60 [ 374.707178] [<ffffffff810882d7>] hrtimer_interrupt+0xe7/0x220 [ 374.736837] [<ffffffff8161b0e9>] smp_apic_timer_interrupt+0x69/0x9c [ 374.768883] [<ffffffff81619fdd>] apic_timer_interrupt+0x6d/0x80 [ 374.799400] <EOI> <NMI> [<ffffffff816048a8>] ? panic+0x193/0x1d7 [ 374.831195] [<ffffffff81604812>] ? panic+0xfd/0x1d7 [ 374.855966] [<ffffffff810f0120>] ? watchdog_enable+0x1e0/0x1e0 [ 374.886109] [<ffffffff810f01e2>] watchdog_overflow_callback+0xc2/0xd0 [ 374.921097] [<ffffffff8112a89e>] __perf_event_overflow+0x8e/0x220 [ 374.955486] [<ffffffff81129729>] ? perf_event_update_userpage+0x19/0x100 [ 374.991471] [<ffffffff8112b484>] perf_event_overflow+0x14/0x20 [ 375.024211] [<ffffffff81028c92>] intel_pmu_handle_irq+0x1b2/0x370 [ 375.057297] [<ffffffff816129ed>] perf_event_nmi_handler+0x1d/0x20 [ 375.089114] [<ffffffff816121b9>] nmi_handle.isra.0+0x59/0x90 [ 375.117439] [<ffffffff816122c0>] do_nmi+0xd0/0x310 [ 375.142167] [<ffffffff81611801>] end_repeat_nmi+0x1e/0x2e [ 375.169249] [<ffffffff81610de0>] ? _raw_spin_lock_irqsave+0x30/0x40 [ 375.201372] [<ffffffff81610de0>] ? _raw_spin_lock_irqsave+0x30/0x40 [ 375.232709] [<ffffffff81610de0>] ? _raw_spin_lock_irqsave+0x30/0x40 [ 375.264788] <<EOE>> [<ffffffff81084f39>] remove_wait_queue+0x19/0x40 [ 375.298191] [<ffffffff811ae9c1>] poll_freewait+0x71/0xa0 [ 375.324866] [<ffffffff811af15a>] do_select+0x5ca/0x650 [ 375.351625] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.382475] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.414047] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.445344] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.479068] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.512964] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.547448] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.580977] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.613662] [<ffffffff811ae760>] ? poll_schedule_timeout+0x70/0x70 [ 375.644540] [<ffffffff811af3d7>] core_sys_select+0x1f7/0x310 [ 375.673468] [<ffffffff814ec7f6>] ? sys_sendto+0x126/0x1c0 [ 375.700596] [<ffffffff8101a309>] ? read_tsc+0x9/0x20 [ 375.726347] [<ffffffff810b2a28>] ? ktime_get_ts+0x48/0xe0 [ 375.753522] [<ffffffff8101a309>] ? read_tsc+0x9/0x20 [ 375.779227] [<ffffffff810b2a28>] ? ktime_get_ts+0x48/0xe0 [ 375.806136] [<ffffffff811af59e>] sys_select+0xae/0x100 [ 375.833004] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 375.863012] ---[ end trace 94a32a2cfce117de ]- or this, [ 7263.377379] XFS (dm-2): Mounting Filesystem [ 7263.491740] XFS (dm-2): Ending clean mount [ 7265.014356] XFS (dm-0): Mounting Filesystem [ 7265.671630] XFS (dm-0): Ending clean mount [ 7266.738912] BUG: unable to handle kernel paging request at ffff8800764e0000 [ 7266.745904] IP: [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 7266.751141] PGD 1ddf067 PUD 1de2067 PMD 76520063 PTE 80000000764e0161 [ 7266.757637] Oops: 0003 [#1] SMP [ 7266.760892] Modules linked in: binfmt_misc(F) snd_seq_dummy(F) tun(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) snd_hda_codec_hdmi(F) snd_hda_codec_realtek(F) snd_hda_intel(F) snd_hda_codec(F) snd_hwdep(F) snd_seq(F) e1000e(F) snd_seq_device(F) snd_pcm(F) hp_wmi(F) iTCO_wdt(F) sparse_keymap(F) iTCO_vendor_support(F) snd_page_alloc(F) rfkill(F) ptp(F) snd_timer(F) mei(F) lpc_ich(F) pps_core(F) snd(F) pcspkr(F) i2c_i801(F) mfd_core(F) microcode(F) soundcore(F) xfs(F) libcrc32c(F) sr_mod(F) sd_mod(F) cdrom(F) crc_t10dif(F) nouveau(F) video(F) mxm_wmi(F) i2c_algo_bit(F) drm_kms_helper(F) a ta_generic(F) ttm(F) ahci(F) pata_acpi(F) drm(F) libahci(F) i2c_core(F) wmi(F) libata(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 7266.860252] CPU 3 [ 7266.862092] Pid: 24364, comm: attr Tainted: GF 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 7266.872091] RIP: 0010:[<ffffffff813022fa>] [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 7266.879756] RSP: 0018:ffff880063573ad8 EFLAGS: 00010282 [ 7266.885057] RAX: ffff88007649c540 RBX: ffff880057665000 RCX: 000000000000d00b [ 7266.892182] RDX: fffffffffffbc4f8 RSI: ffff8800764e03f8 RDI: ffff8800764e0000 [ 7266.899302] RBP: ffff880063573b30 R08: b7fffffbfeffff47 R09: 530000008efffffc [ 7266.906422] R10: 0affff42d0000000 R11: 8ffffffc16ffff42 R12: ffff88007649c520 [ 7266.913549] R13: 00000000000003d8 R14: 0000000000001800 R15: 000000000000007b [ 7266.920668] FS: 00007f1b4a778740(0000) GS:ffff88007d180000(0000) knlGS:0000000000000000 [ 7266.928750] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7266.934484] CR2: ffff8800764e0000 CR3: 000000006478c000 CR4: 00000000000407e0 [ 7266.941607] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 7266.948734] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 7266.955853] Process attr (pid: 24364, threadinfo ffff880063572000, task ffff880062621ac0) [ 7266.964020] Stack: [ 7266.966029] ffffffffa028b1a1 0000000000000000 0000000000000000 ffff88007649c540 [ 7266.973478] ffff88007649c520 0000000000000018 ffff88007649c520 0000000000000000 [ 7266.980928] ffff88005f1fb7b0 ffff88003e8a8840 ffff88005f9476a0 ffff880063573b68 [ 7266.988376] Call Trace: [ 7266.990846] [<ffffffffa028b1a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [ 7266.998425] [<ffffffffa028b467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [ 7267.005138] [<ffffffffa028ca2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [ 7267.011502] [<ffffffffa0288850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [ 7267.018207] [<ffffffffa0296d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [ 7267.024745] [<ffffffffa028978c>] xfs_attr_set_int+0x30c/0x420 [xfs] [ 7267.031091] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [ 7267.036326] [<ffffffffa0289d1f>] xfs_attr_set+0x7f/0x90 [xfs] [ 7267.042163] [<ffffffffa027fa12>] xfs_xattr_set+0x42/0x70 [xfs] [ 7267.048073] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [ 7267.053728] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [ 7267.059903] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [ 7267.065210] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [ 7267.070348] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [ 7267.076263] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [ 7267.081833] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [ 7267.086885] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [ 7267.092890] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [ 7267.098807] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [ 7267.104206] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 7267.110205] Code: 00 00 48 81 fa a8 02 00 00 72 05 40 38 fe 74 41 48 83 ea 20 48 83 ea 20 4c 8b 1e 4c 8b 56 08 4c 8b 4e 10 4c 8b 46 18 48 8d 76 20 <4c> 89 1f 4c 89 57 08 4c 89 4f 10 4c 89 47 18 48 8d 7f 20 73 d4 [ 7267.130148] RIP [<ffffffff813022fa>] memmove+0x4a/0x1a0 [ 7267.135476] RSP <ffff880063573ad8> [ 7267.138959] CR2: ffff8800764e0000 [ 7267.142290] general protection fault: 0000 [#2] SMP [ 7267.147285] Modules linked in: binfmt_misc(F) snd_seq_dummy(F) tun(F) lockd(F) sunrpc(F) nf_conntrack_netbios_ns(F) nf_conntrack_broadcast(F) ipt_MASQUERADE(F) ip6table_nat(F) nf_nat_ipv6(F) ip6table_mangle(F) ip6t_REJECT(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) iptable_mangle(F) ipt_REJECT(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ebtable_filter(F) ebtables(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) sg(F) snd_hda_codec_hdmi(F) snd_hda_codec_realtek(F) snd_hda_intel(F) snd_hda_codec(F) snd_hwdep(F) snd_seq(F) e1000e(F) snd_seq_device(F) snd_pcm(F) hp_wmi(F) iTCO_wdt(F) sparse_keymap(F) iTCO_vendor_support(F) snd_page_alloc(F) rfkill(F) ptp(F) snd_timer(F) mei(F) lpc_ich(F) pps_core(F) snd(F) pcspkr(F) i2c_i801(F) mfd_core(F) microcode(F) soundcore(F) xfs(F) libcrc32c(F) sr_mod(F) sd_mod(F) cdrom(F) crc_t10dif(F) nouveau(F) video(F) mxm_wmi(F) i2c_algo_bit(F) drm_kms_helper(F) a ta_generic(F) ttm(F) ahci(F) pata_acpi(F) drm(F) libahci(F) i2c_core(F) wmi(F) libata(F) dm_mirror(F) dm_region_hash(F) dm_log(F) dm_mod(F) [ 7267.246679] CPU 3 [ 7267.248519] Pid: 24364, comm: attr Tainted: GF 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 7267.258522] RIP: 0010:[<ffffffffa01c4da8>] [<ffffffffa01c4da8>] nv50_crtc_lut_load+0x98/0x110 [nouveau] [ 7267.268031] RSP: 0018:ffff8800635734a0 EFLAGS: 00010046 [ 7267.273330] RAX: ffff880036e14908 RBX: ffffffff810a3297 RCX: 0000000000000000 [ 7267.280450] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffc90010414000 [ 7267.287570] RBP: ffff8800635734b8 R08: ffffc90010414000 R09: 0000000000000000 [ 7267.294697] R10: 8b4cc28948c08548 R11: 000000000000ffff R12: ffffc90010414000 [ 7267.301817] R13: ffff880036e14520 R14: ffff880076abfb10 R15: ffff880076abfbb0 [ 7267.308938] FS: 00007f1b4a778740(0000) GS:ffff88007d180000(0000) knlGS:0000000000000000 [ 7267.317017] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7267.322750] CR2: ffff8800764e0000 CR3: 000000006478c000 CR4: 00000000000407e0 [ 7267.329869] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 7267.336987] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 7267.344115] Process attr (pid: 24364, threadinfo ffff880063572000, task ffff880062621ac0) [ 7267.352280] Stack: [ 7267.354289] ffff880075c68000 000000000000000f 0000000000000010 ffff880063573570 [ 7267.361737] ffffffffa00e62fb ffff8800635735f8 ffffffff8133fa06 0000000000000010 [ 7267.369186] ffff880075c70320 ffff880075c68000 ffffffffa01de540 ffff880076abfb10 [ 7267.376641] Call Trace: [ 7267.379089] [<ffffffffa00e62fb>] drm_fb_helper_setcmap+0x2fb/0x3c0 [drm_kms_helper] [ 7267.386822] [<ffffffff8133fa06>] ? fb_set_var+0x276/0x430 [ 7267.392305] [<ffffffff8160dd32>] ? mutex_lock+0x12/0x30 [ 7267.397613] [<ffffffff8134268a>] fb_set_cmap+0x6a/0x140 [ 7267.402923] [<ffffffff8134ad9c>] fbcon_set_palette+0x13c/0x170 [ 7267.408838] [<ffffffff8134dd85>] fbcon_switch+0x3b5/0x550 [ 7267.414325] [<ffffffff813becc9>] redraw_screen+0x189/0x260 [ 7267.419891] [<ffffffff8133f70e>] ? fb_blank+0x9e/0xc0 [ 7267.425027] [<ffffffff8134c4ca>] fbcon_blank+0x20a/0x2d0 [ 7267.430425] [<ffffffff810898c2>] ? up+0x32/0x50 [ 7267.435039] [<ffffffff81062e84>] ? wake_up_klogd+0x34/0x50 [ 7267.440609] [<ffffffff810701f3>] ? __internal_add_timer+0x113/0x130 [ 7267.446953] [<ffffffff81070847>] ? internal_add_timer+0x17/0x40 [ 7267.452956] [<ffffffff81071b4f>] ? mod_timer+0x10f/0x220 [ 7267.458354] [<ffffffff813bf374>] do_unblank_screen+0xb4/0x1e0 [ 7267.464181] [<ffffffff813bf4b0>] unblank_screen+0x10/0x20 [ 7267.469660] [<ffffffff813034f9>] bust_spinlocks+0x19/0x40 [ 7267.475144] [<ffffffff81611fc4>] oops_end+0x34/0xe0 [ 7267.480103] [<ffffffff81604212>] no_context+0x263/0x270 [ 7267.485410] [<ffffffff81604292>] __bad_area_nosemaphore+0x73/0x1cc [ 7267.491671] [<ffffffff816043fe>] bad_area_nosemaphore+0x13/0x15 [ 7267.497674] [<ffffffff81614c0e>] __do_page_fault+0x31e/0x510 [ 7267.503434] [<ffffffffa0283497>] ? kmem_alloc+0x67/0xf0 [xfs] [ 7267.509266] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [ 7267.515092] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [ 7267.520402] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [ 7267.525799] [<ffffffff81614e0e>] do_page_fault+0xe/0x10 [ 7267.531109] [<ffffffff81611498>] page_fault+0x28/0x30 [ 7267.536246] [<ffffffff813022fa>] ? memmove+0x4a/0x1a0 [ 7267.541398] [<ffffffffa028b1a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [ 7267.548972] [<ffffffffa028b467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [ 7267.555676] [<ffffffffa028ca2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [ 7267.562030] [<ffffffffa0288850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [ 7267.568734] [<ffffffffa0296d4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [ 7267.575264] [<ffffffffa028978c>] xfs_attr_set_int+0x30c/0x420 [xfs] [ 7267.581610] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [ 7267.586845] [<ffffffffa0289d1f>] xfs_attr_set+0x7f/0x90 [xfs] [ 7267.592685] [<ffffffffa027fa12>] xfs_xattr_set+0x42/0x70 [xfs] [ 7267.598595] [<ffffffff811bdef2>] generic_setxattr+0x62/0x80 [ 7267.604252] [<ffffffff811be743>] __vfs_setxattr_noperm+0x63/0x1b0 [ 7267.610429] [<ffffffff811be945>] vfs_setxattr+0xb5/0xc0 [ 7267.615738] [<ffffffff811bea76>] setxattr+0x126/0x1c0 [ 7267.620875] [<ffffffff8118358d>] ? kmem_cache_free+0x1cd/0x1e0 [ 7267.626791] [<ffffffff811a72b2>] ? final_putname+0x22/0x50 [ 7267.632361] [<ffffffff811a74cb>] ? putname+0x2b/0x40 [ 7267.637412] [<ffffffff811ab96f>] ? user_path_at_empty+0x5f/0x90 [ 7267.643415] [<ffffffff8119e5c9>] ? __sb_start_write+0x49/0x100 [ 7267.649331] [<ffffffff811bedef>] sys_lsetxattr+0x8f/0xd0 [ 7267.654727] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 7267.660726] Code: c1 01 48 83 c0 02 48 83 c7 08 49 83 c0 20 41 81 f9 00 01 00 00 74 4c 4c 8b 13 0f b7 30 0f b7 88 00 02 00 00 0f b7 90 00 04 00 00 <4d> 8b 12 66 c1 ee 02 66 c1 e9 02 66 c1 ea 02 66 41 81 3a 6f 90 [ 7267.680665] RIP [<ffffffffa01c4da8>] nv50_crtc_lut_load+0x98/0x110 [nouveau] [ 7267.687835] RSP <ffff8800635734a0> [ 7267.691324] ---[ end trace 0ac6265371f9a5bf ]--- [ 7287.146356] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 3 [ 7288.185689] Shutting down cpus with NMI [ 7288.189526] drm_kms_helper: panic occurred, switching back to text console CAI Qian _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-06-03 3:04 ` CAI Qian @ 2013-06-03 4:00 ` Dave Chinner 2013-06-04 3:14 ` CAI Qian 0 siblings, 1 reply; 18+ messages in thread From: Dave Chinner @ 2013-06-03 4:00 UTC (permalink / raw) To: CAI Qian; +Cc: linux-mm, LKML, stable, xfs On Sun, Jun 02, 2013 at 11:04:11PM -0400, CAI Qian wrote: > > > There's memory corruption all over the place. It is most likely > > that trinity is causing this - it's purpose is to trigger corruption > > issues, but they aren't always immediately seen. If you can trigger > > this xfs trace without trinity having been run and without all the > > RCU/idle/scheduler/cgroup issues occuring at the same time, then > > it's likely to be caused by XFS. But right now, I'd say XFS is just > > an innocent bystander caught in the crossfire. There's nothing I can > > do from an XFS persepctive to track this down... > OK, this can be reproduced by just running LTP and then xfstests without > trinity at all... Cai, can you be more precise about what is triggering it? LTP and xfstests do a large amount of stuff, and stack traces do not do not help narrow down the cause at all. Can you provide the follwoing information and perform the follwoing steps: 1. What xfstest is tripping over it? 2. Can you reproduce it just by running that one specific test on a pristine system (i.e. freshly mkfs'd filesystems, immediately after boot) 3. if you can't reproduce it like that, does it reproduce on an xfstest run on a pristine system? If so, what command line are you running, and what are the filesystem configurations? 4. if you cannot reproduce it just with xfstests and you need to run LTP first, then can you just run the xfstest that is failing after running LTP and see if that triggers the problem. If it does, please take a metadump of the filesystems after LTP has run, save them, and if the single test then fails send me the metadumps and your xfstests command line. 5. If all else fails, bisect the kernel to identify the commit that introduces the problem.... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-06-03 4:00 ` Dave Chinner @ 2013-06-04 3:14 ` CAI Qian 2013-06-04 4:16 ` Dave Chinner 0 siblings, 1 reply; 18+ messages in thread From: CAI Qian @ 2013-06-04 3:14 UTC (permalink / raw) To: Dave Chinner; +Cc: linux-mm, LKML, stable, xfs ----- Original Message ----- > From: "Dave Chinner" <david@fromorbit.com> > To: "CAI Qian" <caiqian@redhat.com> > Cc: xfs@oss.sgi.com, stable@vger.kernel.org, "LKML" <linux-kernel@vger.kernel.org>, "linux-mm" <linux-mm@kvack.org> > Sent: Monday, June 3, 2013 12:00:38 PM > Subject: Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) > > On Sun, Jun 02, 2013 at 11:04:11PM -0400, CAI Qian wrote: > > > > > There's memory corruption all over the place. It is most likely > > > that trinity is causing this - it's purpose is to trigger corruption > > > issues, but they aren't always immediately seen. If you can trigger > > > this xfs trace without trinity having been run and without all the > > > RCU/idle/scheduler/cgroup issues occuring at the same time, then > > > it's likely to be caused by XFS. But right now, I'd say XFS is just > > > an innocent bystander caught in the crossfire. There's nothing I can > > > do from an XFS persepctive to track this down... > > OK, this can be reproduced by just running LTP and then xfstests without > > trinity at all... > > Cai, can you be more precise about what is triggering it? LTP and > xfstests do a large amount of stuff, and stack traces do not do not > help narrow down the cause at all. Can you provide the follwoing > information and perform the follwoing steps: > > 1. What xfstest is tripping over it? Test #20. > 2. Can you reproduce it just by running that one specific test > on a pristine system (i.e. freshly mkfs'd filesystems, > immediately after boot) Yes, it was reproduced without LTP at all. [ 98.534402] XFS (dm-0): Mounting Filesystem [ 98.586673] XFS (dm-0): Ending clean mount [ 99.741704] XFS (dm-2): Mounting Filesystem [ 100.117248] XFS (dm-2): Ending clean mount [ 100.723228] XFS (dm-0): Mounting Filesystem [ 100.775965] XFS (dm-0): Ending clean mount [ 101.980250] BUG: unable to handle kernel NULL pointer dereference at 0000000000000098 [ 101.988136] IP: [<ffffffff81098cac>] tg_load_down+0x4c/0x80 [ 101.993737] PGD 0 [ 101.995769] Oops: 0002 [#1] SMP [ 101.999038] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm hp_wmi sparse_keymap rfkill iTCO_wdt e1000e pcspkr snd_page_alloc iTCO_vendor_support mei ptp pps_core lpc_ich i2c_i801 snd_timer mfd_core microcode(+) snd soundcore xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif nouveau video mxm_wmi i2c_algo_bit drm_kms_helper ahci ata_generic ttm libahci pata_acpi drm i2c_core libata wmi dm_mirror dm_region_hash dm_log dm_mod [ 102.075355] CPU 2 [ 102.077197] Pid: 356, comm: kworker/2:2 Not tainted 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 102.086691] RIP: 0010:[<ffffffff81098cac>] [<ffffffff81098cac>] tg_load_down+0x4c/0x80 [ 102.094705] RSP: 0018:ffff880078307c78 EFLAGS: 00010002 [ 102.100020] RAX: 0001f2b5a618ed0f RBX: 0000000000000001 RCX: 000000000000068a [ 102.107157] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8800772ceee8 [ 102.114293] RBP: ffff880078307c78 R08: 0000000000000008 R09: ffff88007d094400 [ 102.121422] R10: 0000000000000344 R11: 0000000000000001 R12: ffffffff81c78560 [ 102.128552] R13: ffffffff8108c460 R14: 0000000000000000 R15: ffff8800772ceee8 [ 102.135682] FS: 0000000000000000(0000) GS:ffff88007d100000(0000) knlGS:0000000000000000 [ 102.143776] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 102.149524] CR2: 0000000000000098 CR3: 00000000018fa000 CR4: 00000000000407e0 [ 102.156654] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 102.163785] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 102.170915] Process kworker/2:2 (pid: 356, threadinfo ffff880078306000, task ffff88007736b580) [ 102.179527] Stack: [ 102.181545] ffff880078307cc0 ffffffff810926b2 ffffffff81098c60 ffff8800772cf008 [ 102.189005] ffff880079b96f00 000000000000069c ffff880079b96ee8 0000000000014400 [ 102.196464] ffff88007d094400 ffff880078307db0 ffffffff8109f773 ffff88007cc10480 [ 102.203920] Call Trace: [ 102.206372] [<ffffffff810926b2>] walk_tg_tree_from+0x32/0xe0 [ 102.212118] [<ffffffff81098c60>] ? task_waking_fair+0x20/0x20 [ 102.217955] [<ffffffff8109f773>] load_balance+0x2a3/0x7d0 [ 102.223444] [<ffffffff8108fa7c>] ? update_rq_clock.part.67+0x1c/0x170 [ 102.229977] [<ffffffff810a0142>] idle_balance+0x182/0x2f0 [ 102.235468] [<ffffffff8160f1ac>] __schedule+0x7bc/0x7d0 [ 102.240786] [<ffffffff8160f1e9>] schedule+0x29/0x70 [ 102.245756] [<ffffffff8107f404>] worker_thread+0x1b4/0x3d0 [ 102.251332] [<ffffffff8107f250>] ? __alloc_workqueue_key+0x500/0x500 [ 102.257777] [<ffffffff81084240>] kthread+0xc0/0xd0 [ 102.262662] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 102.268761] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [ 102.274164] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 102.280262] Code: 00 00 00 00 48 8b 14 f0 48 8b 0c f1 48 8b 82 98 00 00 00 48 0f af 01 48 8b 0a 31 d2 48 83 c1 01 48 f7 f1 48 8b 57 48 4a 8b 14 02 <48> 89 82 98 00 00 00 31 c0 5d c3 66 0f 1f 84 00 00 00 00 00 48 [ 102.300226] RIP [<ffffffff81098cac>] tg_load_down+0x4c/0x80 [ 102.305906] RSP <ffff880078307c78> [ 102.309394] CR2: 0000000000000098 [ 102.312710] ---[ end trace ba964230a74993fe ]--- [ 102.312714] BUG: unable to handle kernel NULL pointer dereference at 0000000000000098 [ 102.312719] IP: [<ffffffff81098cac>] tg_load_down+0x4c/0x80 [ 102.312720] PGD 0 [ 102.312723] Oops: 0002 [#2] SMP [ 102.312752] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm hp_wmi sparse_keymap rfkill iTCO_wdt e1000e pcspkr snd_page_alloc iTCO_vendor_support mei ptp pps_core lpc_ich i2c_i801 snd_timer mfd_core microcode(+) snd soundcore xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif nouveau video mxm_wmi i2c_algo_bit drm_kms_helper ahci ata_generic ttm libahci pata_acpi drm i2c_core libata wmi dm_mirror dm_region_hash dm_log dm_mod [ 102.312758] CPU 0 [ 102.312758] Pid: 78, comm: kworker/0:2 Tainted: G D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 102.312762] RIP: 0010:[<ffffffff81098cac>] [<ffffffff81098cac>] tg_load_down+0x4c/0x80 [ 102.312763] RSP: 0018:ffff880036c25c78 EFLAGS: 00010002 [ 102.312764] RAX: 0001f2b5a618ed0f RBX: 0000000000000001 RCX: 000000000000068a [ 102.312765] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8800772ceee8 [ 102.312766] RBP: ffff880036c25c78 R08: 0000000000000008 R09: ffff88007d094400 [ 102.312767] R10: 0000000000000344 R11: 0000000000000001 R12: ffffffff81c78560 [ 102.312768] R13: ffffffff8108c460 R14: 0000000000000000 R15: ffff8800772ceee8 [ 102.312770] FS: 0000000000000000(0000) GS:ffff88007d000000(0000) knlGS:0000000000000000 [ 102.312771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 102.312772] CR2: 0000000000000098 CR3: 00000000018fa000 CR4: 00000000000407f0 [ 102.312773] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 102.312774] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 102.312776] Process kworker/0:2 (pid: 78, threadinfo ffff880036c24000, task ffff8800773c9ac0) [ 102.312776] Stack: [ 102.312779] ffff880036c25cc0 ffffffff810926b2 ffffffff81098c60 ffff8800772cf008 [ 102.312781] ffff880079b96f00 000000000000069c ffff880079b96ee8 0000000000014400 [ 102.312783] ffff88007d094400 ffff880036c25db0 ffffffff8109f773 ffff88007cc10080 [ 102.312783] Call Trace: [ 102.312787] [<ffffffff810926b2>] walk_tg_tree_from+0x32/0xe0 [ 102.312789] [<ffffffff81098c60>] ? task_waking_fair+0x20/0x20 [ 102.312792] [<ffffffff8109f773>] load_balance+0x2a3/0x7d0 [ 102.312795] [<ffffffff8108fa7c>] ? update_rq_clock.part.67+0x1c/0x170 [ 102.312798] [<ffffffff810a0142>] idle_balance+0x182/0x2f0 [ 102.312801] [<ffffffff8160f1ac>] __schedule+0x7bc/0x7d0 [ 102.312803] [<ffffffff8160f1e9>] schedule+0x29/0x70 [ 102.312806] [<ffffffff8107f404>] worker_thread+0x1b4/0x3d0 [ 102.312809] [<ffffffff8107f250>] ? __alloc_workqueue_key+0x500/0x500 [ 102.312811] [<ffffffff81084240>] kthread+0xc0/0xd0 [ 102.312813] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 102.312816] [<ffffffff816192ac>] ret_from_fork+0x7c/0xb0 [ 102.312818] [<ffffffff81084180>] ? insert_kthread_work+0x40/0x40 [ 102.312837] Code: 00 00 00 00 48 8b 14 f0 48 8b 0c f1 48 8b 82 98 00 00 00 48 0f af 01 48 8b 0a 31 d2 48 83 c1 01 48 f7 f1 48 8b 57 48 4a 8b 14 02 <48> 89 82 98 00 00 00 31 c0 5d c3 66 0f 1f 84 00 00 00 00 00 48 [ 102.312839] RIP [<ffffffff81098cac>] tg_load_down+0x4c/0x80 [ 102.312840] RSP <ffff880036c25c78> [ 102.312840] CR2: 0000000000000098 [ 102.312842] ---[ end trace ba964230a74993ff ]--- [ 102.312866] general protection fault: 0000 [#3] SMP [ 102.312896] Modules linked in: lockd sunrpc nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm hp_wmi sparse_keymap rfkill iTCO_wdt e1000e pcspkr snd_page_alloc iTCO_vendor_support mei ptp pps_core lpc_ich i2c_i801 snd_timer mfd_core microcode(+) snd soundcore xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif nouveau video mxm_wmi i2c_algo_bit drm_kms_helper ahci ata_generic ttm libahci pata_acpi drm i2c_core libata wmi dm_mirror dm_region_hash dm_log dm_mod [ 102.312903] CPU 1 [ 102.312903] Pid: 1999, comm: attr Tainted: G D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h [ 102.312908] RIP: 0010:[<ffffffff810983a9>] [<ffffffff810983a9>] irqtime_account_process_tick.isra.2+0x239/0x3c0 [ 102.312909] ============================================================================= [ 102.312910] RSP: 0018:ffff88007d083e08 EFLAGS: 00010003 [ 102.312912] BUG kmalloc-1024 (Tainted: G D ): Padding overwritten. 0xffff88005b4e7ec0-0xffff88005b4e7fff [ 102.312913] RAX: ffff88005b656288 RBX: ffff880079b43c80 RCX: 00000000000000a7 [ 102.312914] ----------------------------------------------------------------------------- [ 102.312914] [ 102.312915] RDX: 6b6b6b6b6b6b6b6b RSI: 0000000000000000 RDI: 0000000000000086 [ 102.312917] INFO: Slab 0xffffea00016d3800 objects=24 used=24 fp=0x (null) flags=0x10000000004080 [ 102.312918] RBP: ffff88007d083e40 R08: 0000000000000000 R09: 0000000225c17d03 [ 102.312921] Pid: 518, comm: in:imklog Tainted: G B D 3.9.4 #1 [ 102.312922] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88007d08e800 [ 102.312922] Call Trace: [ 102.312923] R13: ffff880058063580 R14: 0000000000000000 R15: ffff88007d094c88 [ 102.312926] FS: 00007f237404a740(0000) GS:ffff88007d080000(0000) knlGS:0000000000000000 [ 102.312931] [<ffffffff81181ed2>] slab_err+0xc2/0xf0 [ 102.312932] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 102.312934] CR2: 00007f2373c352a0 CR3: 0000000063ea9000 CR4: 00000000000407e0 [ 102.312937] [<ffffffff81018018>] ? write_ok_or_segv+0x88/0x90 [ 102.312938] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 102.312939] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 102.312943] [<ffffffff81020c2b>] ? save_stack_trace+0x2b/0x50 [ 102.312944] Process attr (pid: 1999, threadinfo ffff88005753e000, task ffff880058063580) [ 102.312945] Stack: [ 102.312947] [<ffffffff81180911>] ? set_track+0x61/0x1b0 [ 102.312949] ffff88007d08e800 [ 102.312952] [<ffffffff8118176d>] ? init_object+0x3d/0x70 [ 102.312953] 0000000000000086 ffff88007d094400 [ 102.312956] [<ffffffff81181ff5>] slab_pad_check.part.41+0xf5/0x170 [ 102.312957] 0000000000014400 [ 102.312967] [<ffffffff81063bdc>] ? do_syslog+0x23c/0x5c0 [ 102.312968] ffff880058063580 0000000000000000 [ 102.312969] [<ffffffff811820e3>] check_slab+0x73/0x100 [ 102.312970] 00000017979e99b3 ffff88007d083e78 [ 102.312970] [ 102.312972] [<ffffffff81606b50>] alloc_debug_processing+0x21/0x118 [ 102.312973] ffffffff8109874c [ 102.312974] [<ffffffff8160772f>] __slab_alloc+0x3b8/0x4a2 [ 102.312974] ffff880058063580 0000000000000000 [ 102.312976] [<ffffffff81063bdc>] ? do_syslog+0x23c/0x5c0 [ 102.312976] 0000000000000001 [ 102.312977] Call Trace: [ 102.312978] [<ffffffff810915aa>] ? finish_task_switch+0xba/0xe0 [ 102.312979] <IRQ> [ 102.312980] [<ffffffff81184dd1>] kmem_cache_alloc_trace+0x1b1/0x200 [ 102.312981] [<ffffffff8109874c>] account_process_tick+0x11c/0x1d0 [ 102.312983] [<ffffffff81063bdc>] do_syslog+0x23c/0x5c0 [ 102.312985] [<ffffffff81071f0d>] update_process_times+0x2d/0x80 [ 102.312986] [<ffffffff810850e0>] ? wake_up_bit+0x30/0x30 [ 102.312988] [<ffffffff810ba1c5>] tick_sched_handle.isra.13+0x25/0x60 [ 102.312991] [<ffffffff8120ca94>] kmsg_read+0x44/0x60 [ 102.312992] [<ffffffff810ba241>] tick_sched_timer+0x41/0x60 [ 102.312994] [<ffffffff811ffa9a>] proc_reg_read+0x6a/0xa0 [ 102.312996] [<ffffffff81087af4>] __run_hrtimer+0x74/0x1d0 [ 102.312997] [<ffffffff8119c56c>] vfs_read+0x9c/0x170 [ 102.312998] [<ffffffff810ba200>] ? tick_sched_handle.isra.13+0x60/0x60 [ 102.313000] [<ffffffff8119c939>] sys_read+0x49/0xa0 [ 102.313001] [<ffffffff810882d7>] hrtimer_interrupt+0xe7/0x220 [ 102.313004] [<ffffffff810e0ef6>] ? __audit_syscall_exit+0x1f6/0x2a0 [ 102.313005] [<ffffffff8161b0e9>] smp_apic_timer_interrupt+0x69/0x9c [ 102.313007] [<ffffffff81619359>] system_call_fastpath+0x16/0x1b [ 102.313008] [<ffffffff81619fdd>] apic_timer_interrupt+0x6d/0x80 [ 102.313009] Padding ffff88005b4e7ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313010] Padding ffff88005b4e7ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313010] <EOI> [ 102.313011] Padding ffff88005b4e7ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313013] [<ffffffff813022f6>] ? memmove+0x46/0x1a0 [ 102.313013] Padding ffff88005b4e7ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 29 01 00 .............).. [ 102.313014] Padding ffff88005b4e7f00: 07 1b 04 73 65 6c 69 6e 75 78 73 79 73 74 65 6d ...selinuxsystem [ 102.313015] Padding ffff88005b4e7f10: 5f 75 3a 6f 62 6a 65 63 74 5f 72 3a 75 73 72 5f _u:object_r:usr_ [ 102.313032] [<ffffffffa02801a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] [ 102.313032] Padding ffff88005b4e7f20: 74 3a 73 30 00 00 00 00 49 4e 81 a4 02 02 00 00 t:s0....IN...... [ 102.313033] Padding ffff88005b4e7f30: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ [ 102.313033] Padding ffff88005b4e7f40: 00 00 00 00 00 00 00 02 51 47 09 00 00 00 00 00 ........QG...... [ 102.313042] [<ffffffffa0280467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] [ 102.313043] Padding ffff88005b4e7f50: 51 47 09 00 00 00 00 00 51 ac 1e 27 21 f1 4e ad QG......Q..'!.N. [ 102.313043] Padding ffff88005b4e7f60: 00 00 00 00 00 00 00 f2 00 00 00 00 00 00 00 01 ................ [ 102.313044] Padding ffff88005b4e7f70: 00 00 00 00 00 00 00 01 00 00 0e 01 00 00 00 00 ................ [ 102.313053] [<ffffffffa0281a2e>] xfs_attr_leaf_add+0xce/0x170 [xfs] [ 102.313053] Padding ffff88005b4e7f80: 00 00 00 00 c1 6d 78 44 ff ff ff ff 00 00 00 00 .....mxD........ [ 102.313054] Padding ffff88005b4e7f90: 00 00 00 00 00 00 08 10 36 a0 00 01 00 00 00 00 ........6....... [ 102.313062] [<ffffffffa027d850>] xfs_attr_leaf_addname+0xc0/0x3d0 [xfs] [ 102.313062] Padding ffff88005b4e7fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313063] Padding ffff88005b4e7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313072] [<ffffffffa028bd4e>] ? xfs_bmap_one_block+0x3e/0xa0 [xfs] [ 102.313072] Padding ffff88005b4e7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313073] Padding ffff88005b4e7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313074] Padding ffff88005b4e7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 102.313081] [<ffffffffa027e78c>] xfs_attr_set_int+0x30c/0x420 [xfs] [ 102.313082] Padding ffff88005b4e7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 29 01 00 .............).. [ 102.313085] [<ffffffff811be9f4>] ? setxattr+0xa4/0x1c0 [ 102.313085] FIX kmalloc-1024: Restoring 0xffff88005b4e7ec0-0xffff88005b4e7fff=0x5a [ 102.313085] [ 102.313086] ========== CAI Qian > 3. if you can't reproduce it like that, does it reproduce on > an xfstest run on a pristine system? If so, what command > line are you running, and what are the filesystem > configurations? > 4. if you cannot reproduce it just with xfstests and you need > to run LTP first, then can you just run the xfstest that > is failing after running LTP and see if that triggers the > problem. If it does, please take a metadump of the > filesystems after LTP has run, save them, and if the > single test then fails send me the metadumps and your > xfstests command line. > 5. If all else fails, bisect the kernel to identify the > commit that introduces the problem.... > > Cheers, > > Dave. > > -- > Dave Chinner > david@fromorbit.com > > -- > To unsubscribe, send a message with 'unsubscribe linux-mm' in > the body to majordomo@kvack.org. For more info on Linux MM, > see: http://www.linux-mm.org/ . > Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> > _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-06-04 3:14 ` CAI Qian @ 2013-06-04 4:16 ` Dave Chinner 2013-06-04 4:29 ` Dave Chinner 2013-06-04 4:57 ` CAI Qian 0 siblings, 2 replies; 18+ messages in thread From: Dave Chinner @ 2013-06-04 4:16 UTC (permalink / raw) To: CAI Qian; +Cc: linux-mm, LKML, stable, xfs On Mon, Jun 03, 2013 at 11:14:56PM -0400, CAI Qian wrote: > > > ----- Original Message ----- > > From: "Dave Chinner" <david@fromorbit.com> > > To: "CAI Qian" <caiqian@redhat.com> > > Cc: xfs@oss.sgi.com, stable@vger.kernel.org, "LKML" <linux-kernel@vger.kernel.org>, "linux-mm" <linux-mm@kvack.org> > > Sent: Monday, June 3, 2013 12:00:38 PM > > Subject: Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) > > > > On Sun, Jun 02, 2013 at 11:04:11PM -0400, CAI Qian wrote: > > > > > > > There's memory corruption all over the place. It is most likely > > > > that trinity is causing this - it's purpose is to trigger corruption > > > > issues, but they aren't always immediately seen. If you can trigger > > > > this xfs trace without trinity having been run and without all the > > > > RCU/idle/scheduler/cgroup issues occuring at the same time, then > > > > it's likely to be caused by XFS. But right now, I'd say XFS is just > > > > an innocent bystander caught in the crossfire. There's nothing I can > > > > do from an XFS persepctive to track this down... > > > OK, this can be reproduced by just running LTP and then xfstests without > > > trinity at all... > > > > Cai, can you be more precise about what is triggering it? LTP and > > xfstests do a large amount of stuff, and stack traces do not do not > > help narrow down the cause at all. Can you provide the follwoing > > information and perform the follwoing steps: > > > > 1. What xfstest is tripping over it? > Test #20. > > 2. Can you reproduce it just by running that one specific test > > on a pristine system (i.e. freshly mkfs'd filesystems, > > immediately after boot) > Yes, it was reproduced without LTP at all. > [ 98.534402] XFS (dm-0): Mounting Filesystem > [ 98.586673] XFS (dm-0): Ending clean mount > [ 99.741704] XFS (dm-2): Mounting Filesystem > [ 100.117248] XFS (dm-2): Ending clean mount > [ 100.723228] XFS (dm-0): Mounting Filesystem > [ 100.775965] XFS (dm-0): Ending clean mount > [ 101.980250] BUG: unable to handle kernel NULL pointer dereference at 0000000000000098 > [ 101.988136] IP: [<ffffffff81098cac>] tg_load_down+0x4c/0x80 first bug, in scheduler > [ 102.312714] BUG: unable to handle kernel NULL pointer dereference at 0000000000000098 > [ 102.312719] IP: [<ffffffff81098cac>] tg_load_down+0x4c/0x80 second bug, in scheduler > [ 102.312842] ---[ end trace ba964230a74993ff ]--- > [ 102.312866] general protection fault: 0000 [#3] SMP Third bug > [ 102.312903] Pid: 1999, comm: attr Tainted: G D 3.9.4 #1 Hewlett-Packard HP Z210 Workstation/1587h > [ 102.312908] RIP: 0010:[<ffffffff810983a9>] [<ffffffff810983a9>] irqtime_account_process_tick.isra.2+0x239/0x3c0 In the timer tick code. > [ 102.312909] ============================================================================= > [ 102.312910] RSP: 0018:ffff88007d083e08 EFLAGS: 00010003 > [ 102.312912] BUG kmalloc-1024 (Tainted: G D ): Padding overwritten. 0xffff88005b4e7ec0-0xffff88005b4e7fff > [ 102.312913] RAX: ffff88005b656288 RBX: ffff880079b43c80 RCX: 00000000000000a7 > [ 102.312914] ----------------------------------------------------------------------------- And a memory overwrite. > [ 102.313009] Padding ffff88005b4e7ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313010] Padding ffff88005b4e7ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313011] Padding ffff88005b4e7ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313013] Padding ffff88005b4e7ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 29 01 00 .............).. > [ 102.313014] Padding ffff88005b4e7f00: 07 1b 04 73 65 6c 69 6e 75 78 73 79 73 74 65 6d ...selinuxsystem > [ 102.313015] Padding ffff88005b4e7f10: 5f 75 3a 6f 62 6a 65 63 74 5f 72 3a 75 73 72 5f _u:object_r:usr_ > [ 102.313032] Padding ffff88005b4e7f20: 74 3a 73 30 00 00 00 00 49 4e 81 a4 02 02 00 00 t:s0....IN...... > [ 102.313033] Padding ffff88005b4e7f30: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ > [ 102.313033] Padding ffff88005b4e7f40: 00 00 00 00 00 00 00 02 51 47 09 00 00 00 00 00 ........QG...... > [ 102.313043] Padding ffff88005b4e7f50: 51 47 09 00 00 00 00 00 51 ac 1e 27 21 f1 4e ad QG......Q..'!.N. > [ 102.313043] Padding ffff88005b4e7f60: 00 00 00 00 00 00 00 f2 00 00 00 00 00 00 00 01 ................ > [ 102.313044] Padding ffff88005b4e7f70: 00 00 00 00 00 00 00 01 00 00 0e 01 00 00 00 00 ................ > [ 102.313053] Padding ffff88005b4e7f80: 00 00 00 00 c1 6d 78 44 ff ff ff ff 00 00 00 00 .....mxD........ > [ 102.313054] Padding ffff88005b4e7f90: 00 00 00 00 00 00 08 10 36 a0 00 01 00 00 00 00 ........6....... > [ 102.313062] Padding ffff88005b4e7fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313063] Padding ffff88005b4e7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313072] Padding ffff88005b4e7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313073] Padding ffff88005b4e7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313074] Padding ffff88005b4e7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 102.313082] Padding ffff88005b4e7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 29 01 00 .............).. Oh, look, that contains attributes, and being at the top of a page, that tallies with the attribute code copying data from the top of the block down.... > [ 102.313013] [<ffffffff813022f6>] ? memmove+0x46/0x1a0 > [ 102.313032] [<ffffffffa02801a1>] ? xfs_attr_leaf_moveents.isra.2+0x91/0x280 [xfs] > [ 102.313042] [<ffffffffa0280467>] xfs_attr_leaf_compact+0xd7/0x130 [xfs] And the attribute code is writing into a block sized buffer that is allocated in xfs_attr_leaf_compact(). So, whatever page got handed to xfs_attr_leaf_compact() is actually referenced by the 1024 byte and supposed to be free. So, did XFS ask for a 1k block, or something else? Cai, I did ask you for the information that would have answered this question: > > 3. if you can't reproduce it like that, does it reproduce on > > an xfstest run on a pristine system? If so, what command > > line are you running, and what are the filesystem > > configurations? So, I need xfstests command line and the xfs_info output from the filesystems in use at the time this problem occurs.. That said, this still looks like something has corrupted the heap and XFS is just tripping over it.... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-06-04 4:16 ` Dave Chinner @ 2013-06-04 4:29 ` Dave Chinner 2013-06-04 4:57 ` CAI Qian 1 sibling, 0 replies; 18+ messages in thread From: Dave Chinner @ 2013-06-04 4:29 UTC (permalink / raw) To: CAI Qian; +Cc: linux-mm, LKML, stable, xfs On Tue, Jun 04, 2013 at 02:16:18PM +1000, Dave Chinner wrote: > On Mon, Jun 03, 2013 at 11:14:56PM -0400, CAI Qian wrote: > > [ 102.312909] ============================================================================= > > [ 102.312910] RSP: 0018:ffff88007d083e08 EFLAGS: 00010003 > > [ 102.312912] BUG kmalloc-1024 (Tainted: G D ): Padding overwritten. 0xffff88005b4e7ec0-0xffff88005b4e7fff > > [ 102.312913] RAX: ffff88005b656288 RBX: ffff880079b43c80 RCX: 00000000000000a7 > > [ 102.312914] ----------------------------------------------------------------------------- > > And a memory overwrite. > > > [ 102.313009] Padding ffff88005b4e7ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313010] Padding ffff88005b4e7ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313011] Padding ffff88005b4e7ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313013] Padding ffff88005b4e7ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 29 01 00 .............).. > > [ 102.313014] Padding ffff88005b4e7f00: 07 1b 04 73 65 6c 69 6e 75 78 73 79 73 74 65 6d ...selinuxsystem > > [ 102.313015] Padding ffff88005b4e7f10: 5f 75 3a 6f 62 6a 65 63 74 5f 72 3a 75 73 72 5f _u:object_r:usr_ > > [ 102.313032] Padding ffff88005b4e7f20: 74 3a 73 30 00 00 00 00 49 4e 81 a4 02 02 00 00 t:s0....IN...... > > [ 102.313033] Padding ffff88005b4e7f30: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ > > [ 102.313033] Padding ffff88005b4e7f40: 00 00 00 00 00 00 00 02 51 47 09 00 00 00 00 00 ........QG...... > > [ 102.313043] Padding ffff88005b4e7f50: 51 47 09 00 00 00 00 00 51 ac 1e 27 21 f1 4e ad QG......Q..'!.N. > > [ 102.313043] Padding ffff88005b4e7f60: 00 00 00 00 00 00 00 f2 00 00 00 00 00 00 00 01 ................ > > [ 102.313044] Padding ffff88005b4e7f70: 00 00 00 00 00 00 00 01 00 00 0e 01 00 00 00 00 ................ > > [ 102.313053] Padding ffff88005b4e7f80: 00 00 00 00 c1 6d 78 44 ff ff ff ff 00 00 00 00 .....mxD........ > > [ 102.313054] Padding ffff88005b4e7f90: 00 00 00 00 00 00 08 10 36 a0 00 01 00 00 00 00 ........6....... > > [ 102.313062] Padding ffff88005b4e7fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313063] Padding ffff88005b4e7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313072] Padding ffff88005b4e7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313073] Padding ffff88005b4e7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313074] Padding ffff88005b4e7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > > [ 102.313082] Padding ffff88005b4e7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 29 01 00 .............).. > > Oh, look, that contains attributes, and being at the top of a page, > that tallies with the attribute code copying data from the top of > the block down.... On second thoughts, I'm not so sure of this now. That actually has an inode core in it (the bit starting from "IN"), so it can't be a piece of code from the attribute compaction. So this piece of memory has been used several times by different things before the overwrite has triggered by the look of it. Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) 2013-06-04 4:16 ` Dave Chinner 2013-06-04 4:29 ` Dave Chinner @ 2013-06-04 4:57 ` CAI Qian 1 sibling, 0 replies; 18+ messages in thread From: CAI Qian @ 2013-06-04 4:57 UTC (permalink / raw) To: Dave Chinner; +Cc: linux-mm, LKML, stable, xfs > Cai, I did ask you for the information that would have answered this > question: > > > > 3. if you can't reproduce it like that, does it reproduce on > > > an xfstest run on a pristine system? If so, what command > > > line are you running, and what are the filesystem > > > configurations? > > So, I need xfstests command line and the xfs_info output from the > filesystems in use at the time this problem occurs.. Here you are. [root@hp-z210-01 xfstests-dev]# a=`grep ' swap' /etc/fstab | cut -f 1 -d ' '` [root@hp-z210-01 xfstests-dev]# b=`grep ' /home' /etc/fstab | cut -f 1 -d ' '` [root@hp-z210-01 xfstests-dev]# swapoff -a [root@hp-z210-01 xfstests-dev]# umount /home [root@hp-z210-01 xfstests-dev]# echo "swap = $a" swap = /dev/mapper/rhel_hp--z210--01-swap [root@hp-z210-01 xfstests-dev]# echo "home = $b" home = /dev/mapper/rhel_hp--z210--01-home [root@hp-z210-01 xfstests-dev]# export TEST_DEV=$a [root@hp-z210-01 xfstests-dev]# export TEST_DIR=/mnt/testarea/test [root@hp-z210-01 xfstests-dev]# export SCRATCH_DEV=$b [root@hp-z210-01 xfstests-dev]# export SCRATCH_MNT=/mnt/testarea/scratch [root@hp-z210-01 xfstests-dev]# mkdir -p /mnt/testarea/test [root@hp-z210-01 xfstests-dev]# mkdir -p /mnt/testarea/scratch [root@hp-z210-01 xfstests-dev]# [root@hp-z210-01 xfstests-dev]# mkfs.xfs -f $a meta-data=/dev/mapper/rhel_hp--z210--01-swap isize=256 agcount=4, agsize=251904 blks = sectsz=512 attr=2, projid32bit=0 data = bsize=4096 blocks=1007616, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@hp-z210-01 xfstests-dev]# mkfs.xfs -f $b meta-data=/dev/mapper/rhel_hp--z210--01-home isize=256 agcount=4, agsize=11701504 blks = sectsz=512 attr=2, projid32bit=0 data = bsize=4096 blocks=46806016, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=4096 blocks=22854, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@hp-z210-01 xfstests-dev]# [root@hp-z210-01 xfstests-dev]# mount /dev/mapper/rhel_hp--z210--01-home /mnt/testarea/scratch [root@hp-z210-01 xfstests-dev]# [root@hp-z210-01 xfstests-dev]# mount /dev/mapper/rhel_hp--z210--01-swap /mnt/testarea/test [root@hp-z210-01 xfstests-dev]# xfs_info $a meta-data=/dev/mapper/rhel_hp--z210--01-swap isize=256 agcount=4, agsize=251904 blks = sectsz=512 attr=2 data = bsize=4096 blocks=1007616, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@hp-z210-01 xfstests-dev]# xfs_info $b meta-data=/dev/mapper/rhel_hp--z210--01-home isize=256 agcount=4, agsize=11701504 blks = sectsz=512 attr=2 data = bsize=4096 blocks=46806016, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=22854, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@hp-z210-01 xfstests-dev]# ./check 20 FSTYP -- xfs (non-debug) PLATFORM -- Linux/x86_64 hp-z210-01 3.9.4 MKFS_OPTIONS -- -f -bsize=4096 /dev/mapper/rhel_hp--z210--01-home MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/mapper/rhel_hp--z210--01-home /mnt/testarea/scratch 020 <crashed immediately...> CAI Qian _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2013-06-04 4:57 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <510292845.4997401.1369279175460.JavaMail.root@redhat.com>
2013-05-23 3:21 ` 3.9.3: Oops running xfstests CAI Qian
2013-05-23 3:51 ` Dave Chinner
2013-05-24 8:52 ` CAI Qian
2013-05-27 5:36 ` Dave Chinner
2013-05-27 6:04 ` CAI Qian
2013-05-28 4:31 ` CAI Qian
2013-05-30 3:45 ` CAI Qian
2013-05-30 5:20 ` Dave Chinner
2013-05-30 5:40 ` CAI Qian
2013-05-31 3:03 ` 3.9.4 Oops running xfstests (WAS Re: 3.9.3: Oops running xfstests) CAI Qian
2013-05-31 5:35 ` Eric Sandeen
2013-05-31 6:04 ` Dave Chinner
2013-06-03 3:04 ` CAI Qian
2013-06-03 4:00 ` Dave Chinner
2013-06-04 3:14 ` CAI Qian
2013-06-04 4:16 ` Dave Chinner
2013-06-04 4:29 ` Dave Chinner
2013-06-04 4:57 ` CAI Qian
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox