From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754314Ab3H3DhJ (ORCPT ); Thu, 29 Aug 2013 23:37:09 -0400 Received: from mx4-phx2.redhat.com ([209.132.183.25]:42558 "EHLO mx4-phx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753446Ab3H3Dgw (ORCPT ); Thu, 29 Aug 2013 23:36:52 -0400 Date: Thu, 29 Aug 2013 23:36:36 -0400 (EDT) From: Zhouping Liu To: linux-mm@kvack.org, LKML Cc: lizefan@huawei.com, Qian CAI , madper Message-ID: <566312294.6919147.1377833796685.JavaMail.root@redhat.com> In-Reply-To: <55340856.6916553.1377832984360.JavaMail.root@redhat.com> Subject: BUG: soft lockup - CPU#25 stuck for 23s! [memcg_process_s:5859] MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.3_GA_5664 (ZimbraWebClient - FF22 (Linux)/8.0.3_GA_5664) Thread-Topic: soft lockup - CPU#25 stuck for 23s! [memcg_process_s:5859] Thread-Index: 85z8UDFc/EFWyUCHV0NgJEOx6HkEEg== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello All, I hit the following errors when running memcg_stress_test.sh comes from LTP test suite on v3.11-rc7+: ------------------------ snip ----------------------------- [ 2163.674483] BUG: soft lockup - CPU#25 stuck for 23s! [memcg_process_s:5859] [ 2163.674489] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2163.674531] CPU: 25 PID: 5859 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2163.674531] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2163.674532] task: ffff884831c99fe0 ti: ffff88483358a000 task.ti: ffff88483358a000 [ 2163.674533] RIP: 0010:[] [] smp_call_function_many+0x25e/0x2c0 [ 2163.674536] RSP: 0000:ffff88483358ba18 EFLAGS: 00000202 [ 2163.674537] RAX: 0000000000000008 RBX: 0000000000000282 RCX: ffff880237c98cb0 [ 2163.674538] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2163.674539] RBP: ffff88483358ba68 R08: ffff882cd089fe00 R09: ffff882cebc17540 [ 2163.674540] R10: ffffea0120ceb200 R11: ffffffff812fc0d9 R12: ffffffff81107cf0 [ 2163.674540] R13: ffff88483358b9b8 R14: 0000000000000206 R15: ffff884831391378 [ 2163.674542] FS: 00007ff5779ec740(0000) GS:ffff882cebc00000(0000) knlGS:0000000000000000 [ 2163.674542] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2163.674543] CR2: 00007f763b3dc469 CR3: 00000017cd6a6000 CR4: 00000000000006e0 [ 2163.674544] Stack: [ 2163.674544] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff88483bcd5200 [ 2163.674556] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2163.674564] 0000000000000019 0000000000000001 ffff88483358ba98 ffffffff810cd87a [ 2163.674571] Call Trace: [ 2163.674573] [] ? drain_pages+0xb0/0xb0 [ 2163.674576] [] ? drain_pages+0xb0/0xb0 [ 2163.674580] [] on_each_cpu_mask+0x2a/0x60 [ 2163.674583] [] drain_all_pages+0xb5/0xc0 [ 2163.674587] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2163.674591] [] alloc_pages_current+0xa9/0x170 [ 2163.674595] [] __page_cache_alloc+0x87/0xb0 [ 2163.674598] [] filemap_fault+0x185/0x400 [ 2163.674602] [] __do_fault+0x71/0x4f0 [ 2163.674605] [] ? load_balance+0x109/0x7e0 [ 2163.674608] [] handle_pte_fault+0x93/0xa40 [ 2163.674612] [] handle_mm_fault+0x291/0x660 [ 2163.674615] [] __do_page_fault+0x146/0x510 [ 2163.674619] [] ? do_nanosleep+0x92/0x130 [ 2163.674623] [] ? hrtimer_nanosleep+0xad/0x170 [ 2163.674626] [] ? hrtimer_get_res+0x50/0x50 [ 2163.674629] [] do_page_fault+0xe/0x10 [ 2163.674633] [] page_fault+0x28/0x30 [ 2163.674635] Code: 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 f6 41 20 01 <75> f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b 75 c0 4c 89 [ 2163.710494] BUG: soft lockup - CPU#26 stuck for 23s! [memcg_process_s:5915] [ 2163.710499] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2163.710543] CPU: 26 PID: 5915 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2163.710543] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2163.710544] task: ffff884a2c42dfa0 ti: ffff884a2e7f0000 task.ti: ffff884a2e7f0000 [ 2163.710545] RIP: 0010:[] [] smp_call_function_many+0x25e/0x2c0 [ 2163.710548] RSP: 0000:ffff884a2e7f1960 EFLAGS: 00000202 [ 2163.710549] RAX: 0000000000000008 RBX: ffff88470bd15210 RCX: ffff880237c98cd8 [ 2163.710550] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2163.710551] RBP: ffff884a2e7f19b0 R08: ffff8846f888fe00 R09: ffff88470bc17540 [ 2163.710552] R10: ffffea011bdd9e00 R11: ffffffff812fc0d9 R12: ffff884a2e7f1918 [ 2163.710552] R13: ffffffff81107cf0 R14: ffff884a2e7f1900 R15: 0000000000000202 [ 2163.710554] FS: 00007f0275b35740(0000) GS:ffff88470bc00000(0000) knlGS:0000000000000000 [ 2163.710554] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2163.710555] CR2: 00007f6df18f169e CR3: 0000002ccee5d000 CR4: 00000000000006e0 [ 2163.710556] Stack: [ 2163.710556] 0000000181a131e0 0000000000015200 ffffffff8114b0b0 ffff88470bd15200 [ 2163.710568] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2163.710576] 000000000000001a 0000000000000001 ffff884a2e7f19e0 ffffffff810cd87a [ 2163.710584] Call Trace: [ 2163.710586] [] ? drain_pages+0xb0/0xb0 [ 2163.710589] [] ? drain_pages+0xb0/0xb0 [ 2163.710593] [] on_each_cpu_mask+0x2a/0x60 [ 2163.710596] [] drain_all_pages+0xb5/0xc0 [ 2163.710600] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2163.710603] [] alloc_pages_current+0xa9/0x170 [ 2163.710607] [] __page_cache_alloc+0x87/0xb0 [ 2163.710611] [] __do_page_cache_readahead+0xf4/0x240 [ 2163.710615] [] ? delayacct_end+0x80/0x90 [ 2163.710618] [] ra_submit+0x21/0x30 [ 2163.710622] [] filemap_fault+0x33c/0x400 [ 2163.710626] [] __do_fault+0x71/0x4f0 [ 2163.710629] [] handle_pte_fault+0x93/0xa40 [ 2163.710633] [] handle_mm_fault+0x291/0x660 [ 2163.710636] [] __do_page_fault+0x146/0x510 [ 2163.710640] [] ? set_next_entity+0x95/0xb0 [ 2163.710643] [] ? __switch_to+0x181/0x4b0 [ 2163.710647] [] ? __schedule+0x3a8/0x7d0 [ 2163.710651] [] do_page_fault+0xe/0x10 [ 2163.710654] [] page_fault+0x28/0x30 [ 2163.710657] Code: 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 f6 41 20 01 <75> f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b 75 c0 4c 89 [ 2163.723515] BUG: soft lockup - CPU#27 stuck for 23s! [systemd:1] [ 2163.723516] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2163.723603] CPU: 27 PID: 1 Comm: systemd Not tainted 3.11.0-rc7+ #1 [ 2163.723605] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2163.723607] task: ffff88022f4d0000 ti: ffff88022f4d8000 task.ti: ffff88022f4d8000 [ 2163.723609] RIP: 0010:[] [] smp_call_function_many+0x25e/0x2c0 [ 2163.723616] RSP: 0000:ffff88022f4d9960 EFLAGS: 00000202 [ 2163.723619] RAX: 0000000000000008 RBX: ffff88470bd55210 RCX: ffff880237c98d00 [ 2163.723620] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2163.723623] RBP: ffff88022f4d99b0 R08: ffff884834897e00 R09: ffff88483bc17540 [ 2163.723625] R10: ffffea0120cc5300 R11: ffffffff812fc0d9 R12: ffff88022f4d9918 [ 2163.723627] R13: ffffffff81107cf0 R14: ffff88022f4d9900 R15: 0000000000000206 [ 2163.723630] FS: 00007feb8ce09880(0000) GS:ffff88483bc00000(0000) knlGS:0000000000000000 [ 2163.723632] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2163.723634] CR2: 0000000000400873 CR3: 00000017cf12f000 CR4: 00000000000006e0 [ 2163.723635] Stack: [ 2163.723636] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff88470bd55200 [ 2163.723657] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2163.723672] 000000000000001b 0000000000000001 ffff88022f4d99e0 ffffffff810cd87a [ 2163.723688] Call Trace: [ 2163.723691] [] ? drain_pages+0xb0/0xb0 [ 2163.723699] [] ? drain_pages+0xb0/0xb0 [ 2163.723706] [] on_each_cpu_mask+0x2a/0x60 [ 2163.723714] [] drain_all_pages+0xb5/0xc0 [ 2163.723722] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2163.723731] [] alloc_pages_current+0xa9/0x170 [ 2163.723739] [] __page_cache_alloc+0x87/0xb0 [ 2163.723747] [] __do_page_cache_readahead+0xf4/0x240 [ 2163.723755] [] ? wake_atomic_t_function+0x40/0x40 [ 2163.723763] [] ra_submit+0x21/0x30 [ 2163.723770] [] filemap_fault+0x33c/0x400 [ 2163.723778] [] __do_fault+0x71/0x4f0 [ 2163.723785] [] handle_pte_fault+0x93/0xa40 [ 2163.723793] [] ? __sigqueue_free.part.15+0x33/0x40 [ 2163.723799] [] ? __dequeue_signal+0x13c/0x220 [ 2163.723806] [] ? recalc_sigpending+0x1b/0x50 [ 2163.723813] [] handle_mm_fault+0x291/0x660 [ 2163.723820] [] __do_page_fault+0x146/0x510 [ 2163.723829] [] ? vfs_read+0xf7/0x170 [ 2163.723837] [] do_page_fault+0xe/0x10 [ 2163.723845] [] page_fault+0x28/0x30 [ 2163.723850] Code: 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 f6 41 20 01 <75> f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b 75 c0 4c 89 [ 2163.779522] BUG: soft lockup - CPU#31 stuck for 23s! [memcg_process_s:5969] [ 2163.779523] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2163.779564] CPU: 31 PID: 5969 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2163.779565] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2163.779566] task: ffff88033432aa80 ti: ffff8803337da000 task.ti: ffff8803337da000 [ 2163.779566] RIP: 0010:[] [] smp_call_function_many+0x25a/0x2c0 [ 2163.779569] RSP: 0000:ffff8803337db960 EFLAGS: 00000202 [ 2163.779570] RAX: 0000000000000008 RBX: ffff88470bc95210 RCX: ffff880237c98da0 [ 2163.779571] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2163.779572] RBP: ffff8803337db9b0 R08: ffff884834894200 R09: ffff88483bc57540 [ 2163.779573] R10: ffffea0120cc2000 R11: ffffffff812fc0d9 R12: ffff8803337db918 [ 2163.779574] R13: ffffffff81107cf0 R14: ffff8803337db900 R15: 0000000000000206 [ 2163.779575] FS: 00007f27a32ec740(0000) GS:ffff88483bc40000(0000) knlGS:0000000000000000 [ 2163.779576] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2163.779576] CR2: 00007f0d65270000 CR3: 00000046f7206000 CR4: 00000000000006e0 [ 2163.779577] Stack: [ 2163.779577] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff88470bc95200 [ 2163.779586] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2163.779594] 000000000000001f 0000000000000001 ffff8803337db9e0 ffffffff810cd87a [ 2163.779602] Call Trace: [ 2163.779603] [] ? drain_pages+0xb0/0xb0 [ 2163.779607] [] ? drain_pages+0xb0/0xb0 [ 2163.779610] [] on_each_cpu_mask+0x2a/0x60 [ 2163.779614] [] drain_all_pages+0xb5/0xc0 [ 2163.779617] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2163.779621] [] alloc_pages_current+0xa9/0x170 [ 2163.779625] [] __page_cache_alloc+0x87/0xb0 [ 2163.779629] [] __do_page_cache_readahead+0xf4/0x240 [ 2163.779632] [] ra_submit+0x21/0x30 [ 2163.779635] [] filemap_fault+0x33c/0x400 [ 2163.779639] [] __do_fault+0x71/0x4f0 [ 2163.779642] [] handle_pte_fault+0x93/0xa40 [ 2163.779646] [] handle_mm_fault+0x291/0x660 [ 2163.779649] [] __do_page_fault+0x146/0x510 [ 2163.779653] [] ? set_next_entity+0x95/0xb0 [ 2163.779656] [] ? __switch_to+0x181/0x4b0 [ 2163.779659] [] ? __schedule+0x3a8/0x7d0 [ 2163.779664] [] do_page_fault+0xe/0x10 [ 2163.779668] [] page_fault+0x28/0x30 [ 2163.779670] Code: 48 63 35 c2 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 41 20 01 75 f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b [ 2167.061460] BUG: soft lockup - CPU#0 stuck for 23s! [memcg_process_s:5891] [ 2167.061461] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2167.061501] CPU: 0 PID: 5891 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2167.061501] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2167.061503] task: ffff884831c98000 ti: ffff884833f1c000 task.ti: ffff884833f1c000 [ 2167.061503] RIP: 0010:[] [] smp_call_function_many+0x25a/0x2c0 [ 2167.061506] RSP: 0000:ffff884833f1d960 EFLAGS: 00000202 [ 2167.061507] RAX: 0000000000000008 RBX: ffff882cebc55210 RCX: ffff880237c979a8 [ 2167.061507] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2167.061508] RBP: ffff884833f1d9b0 R08: ffff88023782aa00 R09: ffff880237c17540 [ 2167.061509] R10: ffffea0008adbb00 R11: ffffffff812fc0d9 R12: ffff884833f1d918 [ 2167.061510] R13: ffffffff81107cf0 R14: ffff884833f1d900 R15: 0000000000000206 [ 2167.061511] FS: 00007fcb5e383740(0000) GS:ffff880237c00000(0000) knlGS:0000000000000000 [ 2167.061512] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2167.061513] CR2: 00007fbc8cce3000 CR3: 00000017cfa5c000 CR4: 00000000000006f0 [ 2167.061513] Stack: [ 2167.061513] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff882cebc55200 [ 2167.061524] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2167.061532] 0000000000000000 0000000000000001 ffff884833f1d9e0 ffffffff810cd87a [ 2167.061540] Call Trace: [ 2167.061542] [] ? drain_pages+0xb0/0xb0 [ 2167.061545] [] ? drain_pages+0xb0/0xb0 [ 2167.061549] [] on_each_cpu_mask+0x2a/0x60 [ 2167.061552] [] drain_all_pages+0xb5/0xc0 [ 2167.061556] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2167.061560] [] alloc_pages_current+0xa9/0x170 [ 2167.061564] [] __page_cache_alloc+0x87/0xb0 [ 2167.061567] [] __do_page_cache_readahead+0xf4/0x240 [ 2167.061571] [] ra_submit+0x21/0x30 [ 2167.061575] [] filemap_fault+0x33c/0x400 [ 2167.061578] [] __do_fault+0x71/0x4f0 [ 2167.061582] [] handle_pte_fault+0x93/0xa40 [ 2167.061585] [] ? change_protection_range+0x665/0x6f0 [ 2167.061589] [] handle_mm_fault+0x291/0x660 [ 2167.061592] [] __do_page_fault+0x146/0x510 [ 2167.061596] [] ? change_protection+0x65/0xb0 [ 2167.061599] [] ? change_prot_numa+0x1b/0x40 [ 2167.061603] [] ? task_numa_work+0x247/0x2e0 [ 2167.061606] [] ? task_work_run+0xac/0xe0 [ 2167.061610] [] do_page_fault+0xe/0x10 [ 2167.061613] [] page_fault+0x28/0x30 [ 2167.061615] Code: 48 63 35 c2 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 41 20 01 75 f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b [ 2167.282517] BUG: soft lockup - CPU#3 stuck for 22s! [memcg_process_s:5993] [ 2167.282518] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2167.282566] CPU: 3 PID: 5993 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2167.282567] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2167.282568] task: ffff8817cf2474e0 ti: ffff8817cb2c2000 task.ti: ffff8817cb2c2000 [ 2167.282569] RIP: 0010:[] [] smp_call_function_many+0x25a/0x2c0 [ 2167.282572] RSP: 0000:ffff8817cb2c3960 EFLAGS: 00000202 [ 2167.282573] RAX: 0000000000000008 RBX: ffff8817ebd55210 RCX: ffff880237c98940 [ 2167.282574] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2167.282575] RBP: ffff8817cb2c39b0 R08: ffff884a2f4b8000 R09: ffff884a3fa17540 [ 2167.282576] R10: ffffea0128b94300 R11: ffffffff812fc0d9 R12: ffff8817cb2c3918 [ 2167.282577] R13: ffffffff81107cf0 R14: ffff8817cb2c3900 R15: 0000000000000202 [ 2167.282578] FS: 00007f7904f75740(0000) GS:ffff884a3fa00000(0000) knlGS:0000000000000000 [ 2167.282579] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2167.282580] CR2: 00007f2d04d17000 CR3: 0000004a2e758000 CR4: 00000000000006e0 [ 2167.282581] Stack: [ 2167.282581] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff8817ebd55200 [ 2167.282591] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2167.282598] 0000000000000003 0000000000000001 ffff8817cb2c39e0 ffffffff810cd87a [ 2167.282606] Call Trace: [ 2167.282608] [] ? drain_pages+0xb0/0xb0 [ 2167.282611] [] ? drain_pages+0xb0/0xb0 [ 2167.282614] [] on_each_cpu_mask+0x2a/0x60 [ 2167.282618] [] drain_all_pages+0xb5/0xc0 [ 2167.282621] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2167.282625] [] alloc_pages_current+0xa9/0x170 [ 2167.282629] [] __page_cache_alloc+0x87/0xb0 [ 2167.282632] [] __do_page_cache_readahead+0xf4/0x240 [ 2167.282636] [] ? wake_atomic_t_function+0x40/0x40 [ 2167.282639] [] ra_submit+0x21/0x30 [ 2167.282642] [] filemap_fault+0x33c/0x400 [ 2167.282646] [] __do_fault+0x71/0x4f0 [ 2167.282649] [] handle_pte_fault+0x93/0xa40 [ 2167.282652] [] handle_mm_fault+0x291/0x660 [ 2167.282656] [] __do_page_fault+0x146/0x510 [ 2167.282659] [] ? set_next_entity+0x95/0xb0 [ 2167.282662] [] ? __switch_to+0x181/0x4b0 [ 2167.282666] [] ? recalc_sigpending+0x1b/0x50 [ 2167.282669] [] ? __set_task_blocked+0x32/0x70 [ 2167.282672] [] do_page_fault+0xe/0x10 [ 2167.282675] [] page_fault+0x28/0x30 [ 2167.282678] Code: 48 63 35 c2 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 41 20 01 75 f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b [ 2167.296528] BUG: soft lockup - CPU#4 stuck for 22s! [memcg_process_s:5819] [ 2167.296528] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2167.296569] CPU: 4 PID: 5819 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2167.296570] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2167.296571] task: ffff880333e68aa0 ti: ffff8803341d0000 task.ti: ffff8803341d0000 [ 2167.296572] RIP: 0010:[] [] smp_call_function_many+0x25a/0x2c0 [ 2167.296575] RSP: 0000:ffff8803341d1a18 EFLAGS: 00000202 [ 2167.296576] RAX: 0000000000000008 RBX: 0000000000000282 RCX: ffff880237c98968 [ 2167.296577] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2167.296577] RBP: ffff8803341d1a68 R08: ffff88022f4f3e00 R09: ffff880237c57540 [ 2167.296578] R10: ffffea0008b39b00 R11: ffffffff812fc0d9 R12: ffffffff81107cf0 [ 2167.296579] R13: ffff8803341d19b8 R14: 0000000000000202 R15: ffff8817cd73e848 [ 2167.296580] FS: 00007f1e30f6a740(0000) GS:ffff880237c40000(0000) knlGS:0000000000000000 [ 2167.296581] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2167.296582] CR2: 00007f9a99796000 CR3: 00000046f5066000 CR4: 00000000000006e0 [ 2167.296582] Stack: [ 2167.296583] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff880237c95200 [ 2167.296593] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2167.296601] 0000000000000004 0000000000000001 ffff8803341d1a98 ffffffff810cd87a [ 2167.296609] Call Trace: [ 2167.296611] [] ? drain_pages+0xb0/0xb0 [ 2167.296614] [] ? drain_pages+0xb0/0xb0 [ 2167.296617] [] on_each_cpu_mask+0x2a/0x60 [ 2167.296621] [] drain_all_pages+0xb5/0xc0 [ 2167.296624] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2167.296628] [] alloc_pages_current+0xa9/0x170 [ 2167.296632] [] __page_cache_alloc+0x87/0xb0 [ 2167.296636] [] filemap_fault+0x185/0x400 [ 2167.296640] [] ? __mem_cgroup_uncharge_common+0x47/0x310 [ 2167.296643] [] __do_fault+0x71/0x4f0 [ 2167.296647] [] ? __frontswap_invalidate_page+0x2b/0x70 [ 2167.296650] [] handle_pte_fault+0x93/0xa40 [ 2167.296654] [] handle_mm_fault+0x291/0x660 [ 2167.296657] [] __do_page_fault+0x146/0x510 [ 2167.296661] [] ? do_nanosleep+0x92/0x130 [ 2167.296665] [] ? hrtimer_nanosleep+0xad/0x170 [ 2167.296668] [] ? hrtimer_get_res+0x50/0x50 [ 2167.296672] [] do_page_fault+0xe/0x10 [ 2167.296675] [] page_fault+0x28/0x30 [ 2167.296677] Code: 48 63 35 c2 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 41 20 01 75 f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b [ 2167.331531] BUG: soft lockup - CPU#5 stuck for 24s! [memcg_process_s:5823] [ 2167.331537] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2167.331577] CPU: 5 PID: 5823 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2167.331578] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2167.331579] task: ffff8839f7070000 ti: ffff8839f7f4e000 task.ti: ffff8839f7f4e000 [ 2167.331580] RIP: 0010:[] [] generic_exec_single+0x7a/0xa0 [ 2167.331583] RSP: 0000:ffff8839f7f4f8a8 EFLAGS: 00000202 [ 2167.331584] RAX: 0000000000000286 RBX: ffffffff811adb59 RCX: 0000000000003d9f [ 2167.331584] RDX: ffff880237c95200 RSI: 0000000000000286 RDI: 0000000000000286 [ 2167.331585] RBP: ffff8839f7f4f8e0 R08: ffff880237c95200 R09: 0000000000000001 [ 2167.331586] R10: 0000000000001f00 R11: 0000000000000000 R12: ffff8839f7f4f830 [ 2167.331587] R13: ffff884a2f4ffbb0 R14: ffffffffa0a6269f R15: ffff8839f7f4f820 [ 2167.331588] FS: 00007f1682c9c740(0000) GS:ffff8817ebc40000(0000) knlGS:0000000000000000 [ 2167.331589] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2167.331590] CR2: 00000000004008c2 CR3: 0000000330daa000 CR4: 00000000000006e0 [ 2167.331590] Stack: [ 2167.331591] 0000000000000286 ffff880237c98af8 0000000000000008 0000000000000005 [ 2167.331602] ffffffff81a10b00 ffffffff8114b0b0 ffffffff81d6ca80 ffff8839f7f4f950 [ 2167.331609] ffffffff810cd335 0000000000000000 ffff88183ffdaba8 ffff880237c98968 [ 2167.331617] Call Trace: [ 2167.331619] [] ? drain_pages+0xb0/0xb0 [ 2167.331622] [] smp_call_function_single+0xe5/0x190 [ 2167.331626] [] ? drain_pages+0xb0/0xb0 [ 2167.331629] [] smp_call_function_many+0x27c/0x2c0 [ 2167.331632] [] ? drain_pages+0xb0/0xb0 [ 2167.331635] [] ? drain_pages+0xb0/0xb0 [ 2167.331638] [] on_each_cpu_mask+0x2a/0x60 [ 2167.331642] [] drain_all_pages+0xb5/0xc0 [ 2167.331645] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2167.331649] [] alloc_pages_current+0xa9/0x170 [ 2167.331653] [] __page_cache_alloc+0x87/0xb0 [ 2167.331656] [] __do_page_cache_readahead+0xf4/0x240 [ 2167.331660] [] ? __mem_cgroup_try_charge+0x411/0xc40 [ 2167.331663] [] ? delayacct_end+0x80/0x90 [ 2167.331667] [] ra_submit+0x21/0x30 [ 2167.331670] [] filemap_fault+0x33c/0x400 [ 2167.331674] [] ? __mem_cgroup_uncharge_common+0x47/0x310 [ 2167.331677] [] __do_fault+0x71/0x4f0 [ 2167.331680] [] ? __frontswap_invalidate_page+0x2b/0x70 [ 2167.331684] [] handle_pte_fault+0x93/0xa40 [ 2167.331687] [] handle_mm_fault+0x291/0x660 [ 2167.331691] [] __do_page_fault+0x146/0x510 [ 2167.331695] [] ? do_nanosleep+0x92/0x130 [ 2167.331698] [] ? hrtimer_nanosleep+0xad/0x170 [ 2167.331701] [] ? hrtimer_get_res+0x50/0x50 [ 2167.331705] [] do_page_fault+0xe/0x10 [ 2167.331708] [] page_fault+0x28/0x30 [ 2167.331710] Code: 89 45 d0 e8 89 50 24 00 48 8b 45 c8 4c 89 ff 48 89 c6 e8 8a fc 53 00 48 3b 5d d0 74 25 45 85 f6 75 09 eb 0f 0f 1f 44 00 00 f3 90 <41> f6 44 24 20 01 75 f6 48 83 c4 10 5b 41 5c 41 5d 41 5e 41 5f [ 2169.410234] nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2169.465328] CPU: 7 PID: 5755 Comm: memcg_process_s Not tainted 3.11.0-rc7+ #1 [ 2169.473172] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2169.480155] task: ffff880333e69540 ti: ffff880333800000 task.ti: ffff880333800000 [ 2169.488379] RIP: 0010:[] [] smp_call_function_many+0x25a/0x2c0 [ 2169.498335] RSP: 0000:ffff880333801a18 EFLAGS: 00000202 [ 2169.504174] RAX: 0000000000000008 RBX: 0000000000000282 RCX: ffff880237c989e0 [ 2169.512016] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2169.519861] RBP: ffff880333801a68 R08: ffff884a2f4bbe00 R09: ffff884a3fa57540 [ 2169.527713] R10: ffffea011bd40900 R11: ffffffff812fc0d9 R12: ffffffff81107cf0 [ 2169.535557] R13: ffff8803338019b8 R14: 0000000000000202 R15: ffff8817cd73e998 [ 2169.543402] FS: 00007f5019e79740(0000) GS:ffff884a3fa40000(0000) knlGS:0000000000000000 [ 2169.552297] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2169.558612] CR2: 00007f7d0a209410 CR3: 00000046f7035000 CR4: 00000000000006e0 [ 2169.566454] Stack: [ 2169.568660] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff88470bd55200 [ 2169.576832] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2169.585006] 0000000000000007 0000000000000001 ffff880333801a98 ffffffff810cd87a [ 2169.593242] Call Trace: [ 2169.595929] [] ? drain_pages+0xb0/0xb0 [ 2169.601865] [] ? drain_pages+0xb0/0xb0 [ 2169.607801] [] on_each_cpu_mask+0x2a/0x60 [ 2169.614024] [] drain_all_pages+0xb5/0xc0 [ 2169.620150] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2169.627136] [] alloc_pages_current+0xa9/0x170 [ 2169.633744] [] __page_cache_alloc+0x87/0xb0 [ 2169.640156] [] filemap_fault+0x185/0x400 [ 2169.646284] [] __do_fault+0x71/0x4f0 [ 2169.652028] [] handle_pte_fault+0x93/0xa40 [ 2169.658346] [] ? set_next_entity+0x95/0xb0 [ 2169.664664] [] handle_mm_fault+0x291/0x660 [ 2169.670982] [] __do_page_fault+0x146/0x510 [ 2169.677300] [] ? do_nanosleep+0x92/0x130 [ 2169.683428] [] ? hrtimer_nanosleep+0xad/0x170 [ 2169.690032] [] ? hrtimer_get_res+0x50/0x50 [ 2169.696350] [] do_page_fault+0xe/0x10 [ 2169.702190] [] page_fault+0x28/0x30 [ 2169.707838] Code: 48 63 35 c2 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 41 20 01 75 f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b [ 2171.209655] BUG: soft lockup - CPU#2 stuck for 23s! [libvirtd:3264] [ 2171.216550] Modules linked in: xt_CHECKSUM tun bridge stp llc ebtable_nat nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg xfs libcrc32c netxen_nic amd64_edac_mod hpilo hpwdt edac_mce_amd sp5100_tco shpchp pcspkr edac_core serio_raw microcode i2c_piix4 acpi_power_meter k10temp acpi_cpufreq mperf radeon sd_mod i2c_algo_bit crc_t10dif drm_kms_helper ttm ata_generic drm pata_acpi ahci libahci pata_atiixp libata hpsa i2c_core dm_mirror dm_region_hash dm_log dm_mod [ 2171.290143] CPU: 2 PID: 3264 Comm: libvirtd Not tainted 3.11.0-rc7+ #1 [ 2171.297318] Hardware name: HP ProLiant DL585 G7, BIOS A16 12/31/2011 [ 2171.304302] task: ffff884831cc5500 ti: ffff884834142000 task.ti: ffff884834142000 [ 2171.312527] RIP: 0010:[] [] smp_call_function_many+0x25e/0x2c0 [ 2171.322484] RSP: 0018:ffff884834143960 EFLAGS: 00000202 [ 2171.328320] RAX: 0000000000000008 RBX: ffff88033bc55210 RCX: ffff880237c98918 [ 2171.336163] RDX: 0000000000000008 RSI: 0000000000000030 RDI: 0000000000000000 [ 2171.344018] RBP: ffff8848341439b0 R08: ffff8839f88abe00 R09: ffff883a0bc17540 [ 2171.351861] R10: ffffea00e7dcac00 R11: ffffffff812fc0d9 R12: ffff884834143918 [ 2171.359704] R13: ffffffff81107cf0 R14: ffff884834143900 R15: 0000000000000206 [ 2171.367546] FS: 00007fa6d47be880(0000) GS:ffff883a0bc00000(0000) knlGS:0000000000000000 [ 2171.376440] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2171.382755] CR2: 00007fde1aace469 CR3: 0000004a2c733000 CR4: 00000000000006e0 [ 2171.390598] Stack: [ 2171.392804] 0000000100000000 0000000000015200 ffffffff8114b0b0 ffff88033bc55200 [ 2171.400973] 0000000000000202 ffffffff81d6ca80 ffffffff8114b0b0 0000000000000000 [ 2171.409144] 0000000000000002 0000000000000001 ffff8848341439e0 ffffffff810cd87a [ 2171.417314] Call Trace: [ 2171.420001] [] ? drain_pages+0xb0/0xb0 [ 2171.425937] [] ? drain_pages+0xb0/0xb0 [ 2171.431872] [] on_each_cpu_mask+0x2a/0x60 [ 2171.438093] [] drain_all_pages+0xb5/0xc0 [ 2171.444220] [] __alloc_pages_nodemask+0x70e/0xa00 [ 2171.451207] [] alloc_pages_current+0xa9/0x170 [ 2171.457811] [] __page_cache_alloc+0x87/0xb0 [ 2171.464224] [] __do_page_cache_readahead+0xf4/0x240 [ 2171.471402] [] ? wake_atomic_t_function+0x40/0x40 [ 2171.478387] [] ra_submit+0x21/0x30 [ 2171.483940] [] filemap_fault+0x33c/0x400 [ 2171.490065] [] __do_fault+0x71/0x4f0 [ 2171.495810] [] handle_pte_fault+0x93/0xa40 [ 2171.502127] [] ? change_protection_range+0x665/0x6f0 [ 2171.509399] [] handle_mm_fault+0x291/0x660 [ 2171.515716] [] __do_page_fault+0x146/0x510 [ 2171.522034] [] ? change_protection+0x65/0xb0 [ 2171.528544] [] ? change_prot_numa+0x1b/0x40 [ 2171.534955] [] ? task_numa_work+0x247/0x2e0 [ 2171.541368] [] ? task_work_run+0xac/0xe0 [ 2171.547495] [] do_page_fault+0xe/0x10 [ 2171.553333] [] page_fault+0x28/0x30 [ 2171.558979] Code: 48 94 00 89 c2 39 f0 0f 8d 2d fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 40 62 a0 81 f6 41 20 01 74 cc 0f 1f 40 00 f3 90 f6 41 20 01 <75> f8 48 63 35 91 48 94 00 eb b7 0f b6 4d b4 48 8b 75 c0 4c 89 [ 2748.377519] nf_conntrack: automatic helper assignment is deprecated and it will be removed soon. Use the iptables CT target to attach helpers instead. ------------------------- snip ----------------------------- How to run the reproducer: 1. Install LTP test suite: $ git clone git://github.com/linux-test-project/ltp.git; make autotools; ./configure; make; make install 2. Execute the reproducer: $ cd /opt/ltp; ./runltp -s memcg_stress_test.sh The issue doesn't occur on any machine, I just tested it on two machine, one machine is OK, which has 16Gb RAM, the other machine is crashed, which has 290G RAM, 8 NUMA nodes. Please let me know if you need more detailed info. -- Thanks, Zhouping