From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ea0-f182.google.com (mail-ea0-f182.google.com [209.85.215.182]) by kanga.kvack.org (Postfix) with ESMTP id 6070B6B0035 for ; Wed, 5 Mar 2014 12:45:22 -0500 (EST) Received: by mail-ea0-f182.google.com with SMTP id b10so1211769eae.27 for ; Wed, 05 Mar 2014 09:45:21 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id z8si5856071eee.125.2014.03.05.09.45.19 for ; Wed, 05 Mar 2014 09:45:20 -0800 (PST) Date: Wed, 5 Mar 2014 12:45:03 -0500 From: Dave Jones Subject: bad rss-counter message in 3.14rc5 Message-ID: <20140305174503.GA16335@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-linux-mm@kvack.org List-ID: To: Linux Kernel Cc: linux-mm@kvack.org, Linus Torvalds , Andrew Morton I just saw this on my box that's been running trinity.. [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) There's nothing else, no trace, nothing. Any ideas where to begin with this? Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wg0-f46.google.com (mail-wg0-f46.google.com [74.125.82.46]) by kanga.kvack.org (Postfix) with ESMTP id 853F46B0035 for ; Wed, 5 Mar 2014 12:57:37 -0500 (EST) Received: by mail-wg0-f46.google.com with SMTP id z12so1699283wgg.29 for ; Wed, 05 Mar 2014 09:57:36 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id oq8si1120184wjc.167.2014.03.05.09.57.34 for ; Wed, 05 Mar 2014 09:57:35 -0800 (PST) Date: Wed, 5 Mar 2014 12:57:25 -0500 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140305175725.GB16335@redhat.com> References: <20140305174503.GA16335@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140305174503.GA16335@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote: > I just saw this on my box that's been running trinity.. > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > There's nothing else, no trace, nothing. Any ideas where to begin with this? ah, on the serial console there was also this truncated warning.. [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) [48924.133273] ------------[ cut here ]------------ [48924.133391] kernel BUG at include/linux/swapops.h:131! Dave 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) 125 { 126 struct page *p = pfn_to_page(swp_offset(entry)); 127 /* 128 * Any use of migration entries may only occur while the 129 * corresponding page is locked 130 */ 131 BUG_ON(!PageLocked(p)); 132 return p; 133 } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ea0-f172.google.com (mail-ea0-f172.google.com [209.85.215.172]) by kanga.kvack.org (Postfix) with ESMTP id 519CD6B0031 for ; Thu, 6 Mar 2014 19:22:23 -0500 (EST) Received: by mail-ea0-f172.google.com with SMTP id l9so2028952eaj.3 for ; Thu, 06 Mar 2014 16:22:22 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id 46si12783719eem.46.2014.03.06.16.22.20 for ; Thu, 06 Mar 2014 16:22:21 -0800 (PST) Date: Thu, 6 Mar 2014 19:22:10 -0500 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140307002210.GA26603@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140305175725.GB16335@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Linux Kernel Cc: linux-mm@kvack.org, Linus Torvalds , Andrew Morton On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote: > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote: > > I just saw this on my box that's been running trinity.. > > > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > > > There's nothing else, no trace, nothing. Any ideas where to begin with this? > > ah, on the serial console there was also this truncated warning.. > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > [48924.133273] ------------[ cut here ]------------ > [48924.133391] kernel BUG at include/linux/swapops.h:131! > > Dave > > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) > 125 { > 126 struct page *p = pfn_to_page(swp_offset(entry)); > 127 /* > 128 * Any use of migration entries may only occur while the > 129 * corresponding page is locked > 130 */ > 131 BUG_ON(!PageLocked(p)); > 132 return p; > 133 } I hit this again, this time a full trace made it over the serial console. This time there was no bad rss-counter message though. kernel BUG at include/linux/swapops.h:131! invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000 RIP: 0010:[] [] migration_entry_to_page.part.47+0x4/0x6 RSP: 0000:ffff88018911bae8 EFLAGS: 00010246 RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26 RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000 FS: 0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0 DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 Stack: ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98 Call Trace: [] unmap_single_vma+0x89e/0x8a0 [] unmap_vmas+0x49/0x90 [] exit_mmap+0xe5/0x1a0 [] mmput+0x73/0x110 [] do_exit+0x2a2/0xb50 [] ? __sigqueue_free.part.11+0x33/0x40 [] ? __dequeue_signal+0x13c/0x220 [] do_group_exit+0x4c/0xc0 [] get_signal_to_deliver+0x2d1/0x6d0 [] do_signal+0x57/0x9d0 [] ? __acct_update_integrals+0x8e/0x120 [] ? preempt_count_sub+0x6b/0xf0 [] ? _raw_spin_unlock+0x31/0x50 [] ? vtime_account_user+0x91/0xa0 [] ? context_tracking_user_exit+0x9b/0x100 [] do_notify_resume+0x71/0xc0 [] retint_signal+0x46/0x90 Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ee0-f50.google.com (mail-ee0-f50.google.com [74.125.83.50]) by kanga.kvack.org (Postfix) with ESMTP id 14A8E6B0031 for ; Mon, 10 Mar 2014 22:49:19 -0400 (EDT) Received: by mail-ee0-f50.google.com with SMTP id c13so3432757eek.37 for ; Mon, 10 Mar 2014 19:49:19 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id t3si38146359eeg.64.2014.03.10.19.49.17 for ; Mon, 10 Mar 2014 19:49:18 -0700 (PDT) Date: Mon, 10 Mar 2014 22:49:06 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311024906.GA9191@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140307002210.GA26603@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton On Thu, Mar 06, 2014 at 07:22:10PM -0500, Dave Jones wrote: > On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote: > > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote: > > > I just saw this on my box that's been running trinity.. > > > > > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > > > > > There's nothing else, no trace, nothing. Any ideas where to begin with this? > > > > ah, on the serial console there was also this truncated warning.. > > > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > [48924.133273] ------------[ cut here ]------------ > > [48924.133391] kernel BUG at include/linux/swapops.h:131! > > > > Dave > > > > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) > > 125 { > > 126 struct page *p = pfn_to_page(swp_offset(entry)); > > 127 /* > > 128 * Any use of migration entries may only occur while the > > 129 * corresponding page is locked > > 130 */ > > 131 BUG_ON(!PageLocked(p)); > > 132 return p; > > 133 } > > I hit this again, this time a full trace made it over the serial console. > This time there was no bad rss-counter message though. > > kernel BUG at include/linux/swapops.h:131! > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000 > RIP: 0010:[] [] migration_entry_to_page.part.47+0x4/0x6 > RSP: 0000:ffff88018911bae8 EFLAGS: 00010246 > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000 > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26 > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000 > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000 > FS: 0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0 > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000 > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 > Stack: > ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff > ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000 > 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98 > Call Trace: > [] unmap_single_vma+0x89e/0x8a0 > [] unmap_vmas+0x49/0x90 > [] exit_mmap+0xe5/0x1a0 > [] mmput+0x73/0x110 > [] do_exit+0x2a2/0xb50 > [] ? __sigqueue_free.part.11+0x33/0x40 > [] ? __dequeue_signal+0x13c/0x220 > [] do_group_exit+0x4c/0xc0 > [] get_signal_to_deliver+0x2d1/0x6d0 > [] do_signal+0x57/0x9d0 > [] ? __acct_update_integrals+0x8e/0x120 > [] ? preempt_count_sub+0x6b/0xf0 > [] ? _raw_spin_unlock+0x31/0x50 > [] ? vtime_account_user+0x91/0xa0 > [] ? context_tracking_user_exit+0x9b/0x100 > [] do_notify_resume+0x71/0xc0 > [] retint_signal+0x46/0x90 > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8 Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug.. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com [209.85.220.41]) by kanga.kvack.org (Postfix) with ESMTP id 59CFE6B0039 for ; Mon, 10 Mar 2014 23:09:17 -0400 (EDT) Received: by mail-pa0-f41.google.com with SMTP id fa1so8201110pad.0 for ; Mon, 10 Mar 2014 20:09:16 -0700 (PDT) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTP id k7si18649228pbl.311.2014.03.10.20.09.15 for ; Mon, 10 Mar 2014 20:09:16 -0700 (PDT) Date: Mon, 10 Mar 2014 20:13:40 -0700 From: Andrew Morton Subject: Re: bad rss-counter message in 3.14rc5 Message-Id: <20140310201340.81994295.akpm@linux-foundation.org> In-Reply-To: <20140311024906.GA9191@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov On Mon, 10 Mar 2014 22:49:06 -0400 Dave Jones wrote: > ... > > > > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) > > > 125 { > > > 126 struct page *p = pfn_to_page(swp_offset(entry)); > > > 127 /* > > > 128 * Any use of migration entries may only occur while the > > > 129 * corresponding page is locked > > > 130 */ > > > 131 BUG_ON(!PageLocked(p)); > > > 132 return p; > > > 133 } > > > > I hit this again, this time a full trace made it over the serial console. > > This time there was no bad rss-counter message though. > > > > kernel BUG at include/linux/swapops.h:131! > > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC > > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore > > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 > > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000 > > RIP: 0010:[] [] migration_entry_to_page.part.47+0x4/0x6 > > RSP: 0000:ffff88018911bae8 EFLAGS: 00010246 > > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000 > > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26 > > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000 > > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c > > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000 > > FS: 0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000 > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0 > > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000 > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 > > Stack: > > ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff > > ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000 > > 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98 > > Call Trace: > > [] unmap_single_vma+0x89e/0x8a0 > > [] unmap_vmas+0x49/0x90 > > [] exit_mmap+0xe5/0x1a0 > > [] mmput+0x73/0x110 > > [] do_exit+0x2a2/0xb50 > > [] ? __sigqueue_free.part.11+0x33/0x40 > > [] ? __dequeue_signal+0x13c/0x220 > > [] do_group_exit+0x4c/0xc0 > > [] get_signal_to_deliver+0x2d1/0x6d0 > > [] do_signal+0x57/0x9d0 > > [] ? __acct_update_integrals+0x8e/0x120 > > [] ? preempt_count_sub+0x6b/0xf0 > > [] ? _raw_spin_unlock+0x31/0x50 > > [] ? vtime_account_user+0x91/0xa0 > > [] ? context_tracking_user_exit+0x9b/0x100 > > [] do_notify_resume+0x71/0xc0 > > [] retint_signal+0x46/0x90 > > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8 > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > while trying to reproduce a different bug.. Damn, I thought we'd fixed that but it seems not. Cc's added. Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) by kanga.kvack.org (Postfix) with ESMTP id 60CC86B003B for ; Tue, 11 Mar 2014 00:41:50 -0400 (EDT) Received: by mail-pa0-f48.google.com with SMTP id hz1so8278903pad.35 for ; Mon, 10 Mar 2014 21:41:50 -0700 (PDT) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTP id m9si18875545pab.32.2014.03.10.21.41.48 for ; Mon, 10 Mar 2014 21:41:49 -0700 (PDT) Date: Mon, 10 Mar 2014 21:46:12 -0700 From: Andrew Morton Subject: Re: bad rss-counter message in 3.14rc5 Message-Id: <20140310214612.3b4de36a.akpm@linux-foundation.org> In-Reply-To: <20140310201340.81994295.akpm@linux-foundation.org> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > while trying to reproduce a different bug.. > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > Guys, what stops the migration target page from coming unlocked in > parallel with zap_pte_range()'s call to migration_entry_to_page()? page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl. I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-f176.google.com (mail-qc0-f176.google.com [209.85.216.176]) by kanga.kvack.org (Postfix) with ESMTP id D66706B003D for ; Tue, 11 Mar 2014 00:50:21 -0400 (EDT) Received: by mail-qc0-f176.google.com with SMTP id m20so8807317qcx.7 for ; Mon, 10 Mar 2014 21:50:21 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id u4si10598478qat.108.2014.03.10.21.50.20 for ; Mon, 10 Mar 2014 21:50:21 -0700 (PDT) Date: Tue, 11 Mar 2014 00:50:05 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311045005.GA12551@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310214612.3b4de36a.akpm@linux-foundation.org> Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > while trying to reproduce a different bug.. > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > Guys, what stops the migration target page from coming unlocked in > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > and page_locked() happen under ptl. > > I don't see any holes in regular migration. Do you know if this is > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? I'll give it an overnight run and let you know tomorrow. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f46.google.com (mail-qa0-f46.google.com [209.85.216.46]) by kanga.kvack.org (Postfix) with ESMTP id 25E5F6B004D for ; Tue, 11 Mar 2014 00:51:19 -0400 (EDT) Received: by mail-qa0-f46.google.com with SMTP id i13so7812600qae.19 for ; Mon, 10 Mar 2014 21:51:18 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id k50si2610832qgf.156.2014.03.10.21.51.18 for ; Mon, 10 Mar 2014 21:51:18 -0700 (PDT) Date: Tue, 11 Mar 2014 00:51:09 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311045109.GB12551@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310214612.3b4de36a.akpm@linux-foundation.org> Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > while trying to reproduce a different bug.. > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > Guys, what stops the migration target page from coming unlocked in > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > and page_locked() happen under ptl. > > I don't see any holes in regular migration. Do you know if this is > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f49.google.com (mail-pa0-f49.google.com [209.85.220.49]) by kanga.kvack.org (Postfix) with ESMTP id 16CE56B005A for ; Tue, 11 Mar 2014 00:57:37 -0400 (EDT) Received: by mail-pa0-f49.google.com with SMTP id lj1so8244811pab.36 for ; Mon, 10 Mar 2014 21:57:36 -0700 (PDT) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTP id k7si18898306pbl.161.2014.03.10.21.57.33 for ; Mon, 10 Mar 2014 21:57:34 -0700 (PDT) Date: Mon, 10 Mar 2014 22:01:58 -0700 From: Andrew Morton Subject: Re: bad rss-counter message in 3.14rc5 Message-Id: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> In-Reply-To: <20140311045109.GB12551@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > while trying to reproduce a different bug.. > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > Guys, what stops the migration target page from coming unlocked in > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > and page_locked() happen under ptl. > > > > I don't see any holes in regular migration. Do you know if this is > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. There probably isn't much point unless trinity is using sys_move_pages(). Is it? If so it would be interesting to disable trinity's move_pages calls and see if it still fails. Grasping at straws here, trying to reduce the amount of code to look at :( -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-f172.google.com (mail-qc0-f172.google.com [209.85.216.172]) by kanga.kvack.org (Postfix) with ESMTP id 419926B0069 for ; Tue, 11 Mar 2014 01:08:03 -0400 (EDT) Received: by mail-qc0-f172.google.com with SMTP id i8so9014573qcq.3 for ; Mon, 10 Mar 2014 22:08:03 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id c10si4122030qcg.81.2014.03.10.22.08.02 for ; Mon, 10 Mar 2014 22:08:02 -0700 (PDT) Date: Tue, 11 Mar 2014 01:07:52 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311050752.GA14329@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote: > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > > while trying to reproduce a different bug.. > > > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > > > Guys, what stops the migration target page from coming unlocked in > > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > > and page_locked() happen under ptl. > > > > > > I don't see any holes in regular migration. Do you know if this is > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > There probably isn't much point unless trinity is using > sys_move_pages(). Is it? Trinity will do every syscall an arch has. In the test case I have so far, I've narrowed it down to the vm group of syscalls (so running with '-g vm' will do anything that I deemed 'vm'. Including.. sys_move_pages) I'll try to narrow it down further tomorrow. > If so it would be interesting to disable > trinity's move_pages calls and see if it still fails. Ok, I'll try that first. > Grasping at straws here, trying to reduce the amount of code to look at :( *nod*, it's not helped by the fact that the trace happens at process exit time which could be considerably later after the syscall that buggers everything up has happened. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-f170.google.com (mail-qc0-f170.google.com [209.85.216.170]) by kanga.kvack.org (Postfix) with ESMTP id 5D3B96B006E for ; Tue, 11 Mar 2014 01:30:30 -0400 (EDT) Received: by mail-qc0-f170.google.com with SMTP id e9so9126855qcy.1 for ; Mon, 10 Mar 2014 22:30:30 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id k1si10642261qaf.53.2014.03.10.22.30.29 for ; Mon, 10 Mar 2014 22:30:29 -0700 (PDT) Date: Tue, 11 Mar 2014 01:30:17 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311053017.GB14329@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote: > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > > while trying to reproduce a different bug.. > > > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > > > Guys, what stops the migration target page from coming unlocked in > > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > > and page_locked() happen under ptl. > > > > > > I don't see any holes in regular migration. Do you know if this is > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > There probably isn't much point unless trinity is using > sys_move_pages(). Is it? If so it would be interesting to disable > trinity's move_pages calls and see if it still fails. Ok, with move_pages excluded it still oopses. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f48.google.com (mail-pb0-f48.google.com [209.85.160.48]) by kanga.kvack.org (Postfix) with ESMTP id 780E66B008A for ; Tue, 11 Mar 2014 08:56:00 -0400 (EDT) Received: by mail-pb0-f48.google.com with SMTP id md12so8728061pbc.7 for ; Tue, 11 Mar 2014 05:56:00 -0700 (PDT) Received: from userp1040.oracle.com (userp1040.oracle.com. [156.151.31.81]) by mx.google.com with ESMTPS id xe9si20157707pab.25.2014.03.11.05.55.59 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 11 Mar 2014 05:55:59 -0700 (PDT) Message-ID: <531F07D4.5000108@oracle.com> Date: Tue, 11 Mar 2014 08:55:48 -0400 From: Sasha Levin MIME-Version: 1.0 Subject: Re: bad rss-counter message in 3.14rc5 References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> In-Reply-To: <20140311053017.GB14329@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Cyrill Gorcunov , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On 03/11/2014 01:30 AM, Dave Jones wrote: > On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote: > > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > > > > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > > > while trying to reproduce a different bug.. > > > > > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > > > > > Guys, what stops the migration target page from coming unlocked in > > > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > > > and page_locked() happen under ptl. > > > > > > > > I don't see any holes in regular migration. Do you know if this is > > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > > > There probably isn't much point unless trinity is using > > sys_move_pages(). Is it? If so it would be interesting to disable > > trinity's move_pages calls and see if it still fails. > > Ok, with move_pages excluded it still oopses. FWIW, yes - I still see both of these issues happening. It's easy to ignore the bad rss-counter, and I've commented out the BUG at swapops.h so that I could keep on testing. There are quite a few issues within mm/ right now, I think there are more than 5 different BUG()s hittable using trinity at this point without a fix. Thanks, Sasha -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f44.google.com (mail-la0-f44.google.com [209.85.215.44]) by kanga.kvack.org (Postfix) with ESMTP id 2BA7F6B0092 for ; Tue, 11 Mar 2014 09:20:28 -0400 (EDT) Received: by mail-la0-f44.google.com with SMTP id hr13so5656067lab.31 for ; Tue, 11 Mar 2014 06:20:27 -0700 (PDT) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com [2a00:1450:4010:c04::22e]) by mx.google.com with ESMTPS id p7si21988457lae.68.2014.03.11.06.20.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 11 Mar 2014 06:20:26 -0700 (PDT) Received: by mail-lb0-f174.google.com with SMTP id u14so5529718lbd.5 for ; Tue, 11 Mar 2014 06:20:25 -0700 (PDT) Date: Tue, 11 Mar 2014 17:20:24 +0400 From: Cyrill Gorcunov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311132024.GC32390@moon> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311053017.GB14329@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote: > > > > > > > > I don't see any holes in regular migration. Do you know if this is > > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > > > There probably isn't much point unless trinity is using > > sys_move_pages(). Is it? If so it would be interesting to disable > > trinity's move_pages calls and see if it still fails. > > Ok, with move_pages excluded it still oopses. Dave, is it possible to somehow figure out was someone reading pagemap file at moment of the bug triggering? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f42.google.com (mail-pa0-f42.google.com [209.85.220.42]) by kanga.kvack.org (Postfix) with ESMTP id CF5D76B0098 for ; Tue, 11 Mar 2014 09:23:16 -0400 (EDT) Received: by mail-pa0-f42.google.com with SMTP id fb1so8882646pad.1 for ; Tue, 11 Mar 2014 06:23:15 -0700 (PDT) Received: from aserp1040.oracle.com (aserp1040.oracle.com. [141.146.126.69]) by mx.google.com with ESMTPS id zo6si20254921pbc.13.2014.03.11.06.23.14 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 11 Mar 2014 06:23:14 -0700 (PDT) Message-ID: <531F0E39.9020100@oracle.com> Date: Tue, 11 Mar 2014 09:23:05 -0400 From: Sasha Levin MIME-Version: 1.0 Subject: Re: bad rss-counter message in 3.14rc5 References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> In-Reply-To: <20140311132024.GC32390@moon> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Cyrill Gorcunov , Dave Jones Cc: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On 03/11/2014 09:20 AM, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote: >> > > > >> > > > I don't see any holes in regular migration. Do you know if this is >> > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? >> > > >> > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. >> > >> > There probably isn't much point unless trinity is using >> > sys_move_pages(). Is it? If so it would be interesting to disable >> > trinity's move_pages calls and see if it still fails. >> >> Ok, with move_pages excluded it still oopses. > > Dave, is it possible to somehow figure out was someone reading pagemap file > at moment of the bug triggering? We can sprinkle printk()s wherever might be useful, might not be 100% accurate but should be close enough to confirm/deny the theory. Thanks, Sasha -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-f177.google.com (mail-lb0-f177.google.com [209.85.217.177]) by kanga.kvack.org (Postfix) with ESMTP id 37E686B009A for ; Tue, 11 Mar 2014 09:42:02 -0400 (EDT) Received: by mail-lb0-f177.google.com with SMTP id z11so5480146lbi.8 for ; Tue, 11 Mar 2014 06:42:01 -0700 (PDT) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [2a00:1450:4010:c04::233]) by mx.google.com with ESMTPS id am6si22083490lbc.18.2014.03.11.06.42.00 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 11 Mar 2014 06:42:00 -0700 (PDT) Received: by mail-lb0-f179.google.com with SMTP id p9so5705517lbv.10 for ; Tue, 11 Mar 2014 06:42:00 -0700 (PDT) Date: Tue, 11 Mar 2014 17:41:58 +0400 From: Cyrill Gorcunov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311134158.GD32390@moon> References: <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <531F0E39.9020100@oracle.com> Sender: owner-linux-mm@kvack.org List-ID: To: Sasha Levin Cc: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: > >> > >>Ok, with move_pages excluded it still oopses. > > > >Dave, is it possible to somehow figure out was someone reading pagemap file > >at moment of the bug triggering? > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but > should be close enough to confirm/deny the theory. After reading some more, I suppose the idea I had is wrong, investigating. Will ping if I find something. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f49.google.com (mail-qa0-f49.google.com [209.85.216.49]) by kanga.kvack.org (Postfix) with ESMTP id C9A506B009F for ; Tue, 11 Mar 2014 10:28:29 -0400 (EDT) Received: by mail-qa0-f49.google.com with SMTP id j7so1019622qaq.8 for ; Tue, 11 Mar 2014 07:28:29 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id s10si10682326qas.115.2014.03.11.07.28.28 for ; Tue, 11 Mar 2014 07:28:28 -0700 (PDT) Date: Tue, 11 Mar 2014 10:28:17 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311142817.GA26517@redhat.com> References: <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311134158.GD32390@moon> Sender: owner-linux-mm@kvack.org List-ID: To: Cyrill Gorcunov Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: > > >> > > >>Ok, with move_pages excluded it still oopses. > > > > > >Dave, is it possible to somehow figure out was someone reading pagemap file > > >at moment of the bug triggering? > > > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but > > should be close enough to confirm/deny the theory. > > After reading some more, I suppose the idea I had is wrong, investigating. > Will ping if I find something. I can rule it out anyway, I can reproduce this by telling trinity to do nothing other than mmap()'s. I'll try and narrow down the exact parameters. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f44.google.com (mail-la0-f44.google.com [209.85.215.44]) by kanga.kvack.org (Postfix) with ESMTP id 12DE86B00A2 for ; Tue, 11 Mar 2014 10:37:58 -0400 (EDT) Received: by mail-la0-f44.google.com with SMTP id hr13so5697606lab.17 for ; Tue, 11 Mar 2014 07:37:58 -0700 (PDT) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [2a00:1450:4010:c03::232]) by mx.google.com with ESMTPS id x2si22221470lad.132.2014.03.11.07.37.56 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 11 Mar 2014 07:37:57 -0700 (PDT) Received: by mail-la0-f50.google.com with SMTP id y1so5805893lam.9 for ; Tue, 11 Mar 2014 07:37:56 -0700 (PDT) Date: Tue, 11 Mar 2014 18:37:50 +0400 From: Cyrill Gorcunov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311143750.GE32390@moon> References: <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311142817.GA26517@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote: > On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote: > > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: > > > >> > > > >>Ok, with move_pages excluded it still oopses. > > > > > > > >Dave, is it possible to somehow figure out was someone reading pagemap file > > > >at moment of the bug triggering? > > > > > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but > > > should be close enough to confirm/deny the theory. > > > > After reading some more, I suppose the idea I had is wrong, investigating. > > Will ping if I find something. > > I can rule it out anyway, I can reproduce this by telling trinity to do nothing > other than mmap()'s. I'll try and narrow down the exact parameters. Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by kanga.kvack.org (Postfix) with ESMTP id 2A95D6B00A4 for ; Tue, 11 Mar 2014 10:58:52 -0400 (EDT) Received: by mail-ob0-f172.google.com with SMTP id wm4so8557678obc.3 for ; Tue, 11 Mar 2014 07:58:51 -0700 (PDT) Received: from aserp1040.oracle.com (aserp1040.oracle.com. [141.146.126.69]) by mx.google.com with ESMTPS id iz10si24522770obb.13.2014.03.11.07.58.51 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 11 Mar 2014 07:58:51 -0700 (PDT) Message-ID: <531F24A6.2020409@oracle.com> Date: Tue, 11 Mar 2014 10:58:46 -0400 From: Sasha Levin MIME-Version: 1.0 Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> In-Reply-To: <20140311143750.GE32390@moon> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Cyrill Gorcunov , Dave Jones Cc: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On 03/11/2014 10:37 AM, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote: >> On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote: >> > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: >> > > >> >> > > >>Ok, with move_pages excluded it still oopses. >> > > > >> > > >Dave, is it possible to somehow figure out was someone reading pagemap file >> > > >at moment of the bug triggering? >> > > >> > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but >> > > should be close enough to confirm/deny the theory. >> > >> > After reading some more, I suppose the idea I had is wrong, investigating. >> > Will ping if I find something. >> >> I can rule it out anyway, I can reproduce this by telling trinity to do nothing >> other than mmap()'s. I'll try and narrow down the exact parameters. > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > was passed, right? Share it too please. I've sent one of those last time I reported this issue: https://lkml.org/lkml/2014/1/22/625 Thanks, Sasha -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) by kanga.kvack.org (Postfix) with ESMTP id 926926B00AE for ; Tue, 11 Mar 2014 13:11:00 -0400 (EDT) Received: by mail-qa0-f44.google.com with SMTP id f11so8701058qae.3 for ; Tue, 11 Mar 2014 10:11:00 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id d8si7327123qao.144.2014.03.11.10.10.59 for ; Tue, 11 Mar 2014 10:11:00 -0700 (PDT) Date: Tue, 11 Mar 2014 13:10:45 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311171045.GA4693@redhat.com> References: <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311143750.GE32390@moon> Sender: owner-linux-mm@kvack.org List-ID: To: Cyrill Gorcunov Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 06:37:50PM +0400, Cyrill Gorcunov wrote: > > > After reading some more, I suppose the idea I had is wrong, investigating. > > > Will ping if I find something. > > > > I can rule it out anyway, I can reproduce this by telling trinity to do nothing > > other than mmap()'s. I'll try and narrow down the exact parameters. > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > was passed, right? Share it too please. Hm, I may have been mistaken, and the damage was done by a previous run. I went from being able to reproduce it almost instantly to now not being able to reproduce it at all. Will keep trying. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f47.google.com (mail-la0-f47.google.com [209.85.215.47]) by kanga.kvack.org (Postfix) with ESMTP id 2517E6B00B2 for ; Tue, 11 Mar 2014 13:36:08 -0400 (EDT) Received: by mail-la0-f47.google.com with SMTP id y1so5800463lam.6 for ; Tue, 11 Mar 2014 10:36:07 -0700 (PDT) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [2a00:1450:4010:c04::232]) by mx.google.com with ESMTPS id oc6si7766938lbb.25.2014.03.11.10.36.05 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 11 Mar 2014 10:36:06 -0700 (PDT) Received: by mail-lb0-f178.google.com with SMTP id s7so5968001lbd.9 for ; Tue, 11 Mar 2014 10:36:05 -0700 (PDT) Date: Tue, 11 Mar 2014 21:36:03 +0400 From: Cyrill Gorcunov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311173603.GG32390@moon> References: <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311171045.GA4693@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > > was passed, right? Share it too please. > > Hm, I may have been mistaken, and the damage was done by a previous run. > I went from being able to reproduce it almost instantly to now not being able > to reproduce it at all. Will keep trying. Sasha already gave a link to the syscalls sequence, so no rush. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-f41.google.com (mail-qg0-f41.google.com [209.85.192.41]) by kanga.kvack.org (Postfix) with ESMTP id 059B16B00B5 for ; Tue, 11 Mar 2014 13:39:28 -0400 (EDT) Received: by mail-qg0-f41.google.com with SMTP id i50so21422958qgf.0 for ; Tue, 11 Mar 2014 10:39:28 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id y4si11608940qad.73.2014.03.11.10.39.28 for ; Tue, 11 Mar 2014 10:39:28 -0700 (PDT) Date: Tue, 11 Mar 2014 13:39:17 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311173917.GB4693@redhat.com> References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311173603.GG32390@moon> Sender: owner-linux-mm@kvack.org List-ID: To: Cyrill Gorcunov Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > > > was passed, right? Share it too please. > > > > Hm, I may have been mistaken, and the damage was done by a previous run. > > I went from being able to reproduce it almost instantly to now not being able > > to reproduce it at all. Will keep trying. > > Sasha already gave a link to the syscalls sequence, so no rush. It'd be nice to get a more concise reproducer, his list had a little of everything in there. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-f178.google.com (mail-lb0-f178.google.com [209.85.217.178]) by kanga.kvack.org (Postfix) with ESMTP id 7A1F36B0072 for ; Fri, 14 Mar 2014 08:27:18 -0400 (EDT) Received: by mail-lb0-f178.google.com with SMTP id s7so1739984lbd.9 for ; Fri, 14 Mar 2014 05:27:17 -0700 (PDT) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [2a00:1450:4010:c04::231]) by mx.google.com with ESMTPS id y10si2974201lad.71.2014.03.14.05.27.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Mar 2014 05:27:16 -0700 (PDT) Received: by mail-lb0-f177.google.com with SMTP id z11so1705476lbi.36 for ; Fri, 14 Mar 2014 05:27:15 -0700 (PDT) Date: Fri, 14 Mar 2014 16:27:14 +0400 From: Cyrill Gorcunov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140314122714.GR13448@moon> References: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311173917.GB4693@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones , Sasha Levin Cc: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 11, 2014 at 01:39:17PM -0400, Dave Jones wrote: > > > > Sasha already gave a link to the syscalls sequence, so no rush. > > It'd be nice to get a more concise reproducer, his list had a little of everything in there. Dave, could you please send me your config privately so I would try to reproduce the issue locally maybe it shed some light on the problem. Cyrill -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f172.google.com (mail-pd0-f172.google.com [209.85.192.172]) by kanga.kvack.org (Postfix) with ESMTP id A9E8B6B0129 for ; Tue, 18 Mar 2014 20:40:05 -0400 (EDT) Received: by mail-pd0-f172.google.com with SMTP id p10so7849910pdj.31 for ; Tue, 18 Mar 2014 17:40:05 -0700 (PDT) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [2607:f8b0:400e:c03::231]) by mx.google.com with ESMTPS id yo5si13552827pab.210.2014.03.18.17.40.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 17:40:04 -0700 (PDT) Received: by mail-pa0-f49.google.com with SMTP id lj1so8080354pab.36 for ; Tue, 18 Mar 2014 17:40:04 -0700 (PDT) Date: Tue, 18 Mar 2014 17:38:38 -0700 (PDT) From: Hugh Dickins Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <20140311173917.GB4693@redhat.com> Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, 11 Mar 2014, Dave Jones wrote: > On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: > > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > > > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > > > > was passed, right? Share it too please. > > > > > > Hm, I may have been mistaken, and the damage was done by a previous run. > > > I went from being able to reproduce it almost instantly to now not being able > > > to reproduce it at all. Will keep trying. > > > > Sasha already gave a link to the syscalls sequence, so no rush. > > It'd be nice to get a more concise reproducer, his list had a little of everything in there. I've so far failed to find any explanation for your swapops.h BUG; but believe I have identified one cause for "Bad rss-counter"s. My hunch is that the swapops.h BUG is "nearby", but I just cannot fit it together (the swapops.h BUG comes when rmap cannot find all all the migration entries it inserted earlier: it's a very useful BUG for validating rmap). Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list. Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc. (Cyrill, entirely unrelated, but in preparing this patch I noticed your soft_dirty work in install_file_pte(): which looked good at first, until I realized that it's propagating the soft_dirty of a pte it's about to zap completely, to the unrelated entry it's about to insert in its place. Which seems very odd to me.) [PATCH] mm: fix bad rss-counter if remap_file_pages raced migration Fix some "Bad rss-counter state" reports on exit, arising from the interaction between page migration and remap_file_pages(): zap_pte() must count a migration entry when zapping it. And yes, it is possible (though very unusual) to find an anon page or swap entry in a VM_SHARED nonlinear mapping: coming from that horrid get_user_pages(write, force) case which COWs even in a shared mapping. Signed-off-by: Hugh Dickins --- mm/fremap.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) --- 3.14-rc7/mm/fremap.c 2014-01-19 18:40:07.000000000 -0800 +++ linux/mm/fremap.c 2014-03-18 16:32:39.288612346 -0700 @@ -23,28 +23,44 @@ #include "internal.h" +static int mm_counter(struct page *page) +{ + return PageAnon(page) ? MM_ANONPAGES : MM_FILEPAGES; +} + static void zap_pte(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { pte_t pte = *ptep; + struct page *page; + swp_entry_t entry; if (pte_present(pte)) { - struct page *page; - flush_cache_page(vma, addr, pte_pfn(pte)); pte = ptep_clear_flush(vma, addr, ptep); page = vm_normal_page(vma, addr, pte); if (page) { if (pte_dirty(pte)) set_page_dirty(page); + update_hiwater_rss(mm); + dec_mm_counter(mm, mm_counter(page)); page_remove_rmap(page); page_cache_release(page); + } + } else { /* zap_pte() is not called when pte_none() */ + if (!pte_file(pte)) { update_hiwater_rss(mm); - dec_mm_counter(mm, MM_FILEPAGES); + entry = pte_to_swp_entry(pte); + if (non_swap_entry(entry)) { + if (is_migration_entry(entry)) { + page = migration_entry_to_page(entry); + dec_mm_counter(mm, mm_counter(page)); + } + } else { + free_swap_and_cache(entry); + dec_mm_counter(mm, MM_SWAPENTS); + } } - } else { - if (!pte_file(pte)) - free_swap_and_cache(pte_to_swp_entry(pte)); pte_clear_not_present_full(mm, addr, ptep, 0); } } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ve0-f176.google.com (mail-ve0-f176.google.com [209.85.128.176]) by kanga.kvack.org (Postfix) with ESMTP id B1EF56B012F for ; Tue, 18 Mar 2014 21:10:41 -0400 (EDT) Received: by mail-ve0-f176.google.com with SMTP id cz12so7887099veb.21 for ; Tue, 18 Mar 2014 18:10:41 -0700 (PDT) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [2607:f8b0:400c:c01::22d]) by mx.google.com with ESMTPS id dm2si4057314vec.111.2014.03.18.18.10.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 18:10:41 -0700 (PDT) Received: by mail-ve0-f173.google.com with SMTP id oy12so8138165veb.32 for ; Tue, 18 Mar 2014 18:10:40 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> Date: Tue, 18 Mar 2014 18:10:40 -0700 Message-ID: Subject: Re: bad rss-counter message in 3.14rc5 From: Linus Torvalds Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins wrote: > > And yes, it is possible (though very unusual) to find an anon page or > swap entry in a VM_SHARED nonlinear mapping: coming from that horrid > get_user_pages(write, force) case which COWs even in a shared mapping. Hmm. Maybe we could just disallow that forced case. It *used* to be a trivial "we can just do a COW", but that was back when the VM was much simpler and we had no rmap's etc. So "that horrid case" used to be a simple hack that wasn't painful. But I suspect we could very easily just fail it instead of forcing a COW, if that would make it simpler for the VM code. Linus -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) by kanga.kvack.org (Postfix) with ESMTP id 5A0AD6B0134 for ; Tue, 18 Mar 2014 21:32:44 -0400 (EDT) Received: by mail-pd0-f171.google.com with SMTP id r10so7921135pdi.2 for ; Tue, 18 Mar 2014 18:32:43 -0700 (PDT) Received: from aserp1040.oracle.com (aserp1040.oracle.com. [141.146.126.69]) by mx.google.com with ESMTPS id po10si13646170pab.97.2014.03.18.18.32.42 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 18 Mar 2014 18:32:43 -0700 (PDT) Message-ID: <5328F3B4.1080208@oracle.com> Date: Tue, 18 Mar 2014 21:32:36 -0400 From: Sasha Levin MIME-Version: 1.0 Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins , Dave Jones Cc: Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On 03/18/2014 08:38 PM, Hugh Dickins wrote: > On Tue, 11 Mar 2014, Dave Jones wrote: >> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: >> > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: >> > > > >> > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence >> > > > was passed, right? Share it too please. >> > > >> > > Hm, I may have been mistaken, and the damage was done by a previous run. >> > > I went from being able to reproduce it almost instantly to now not being able >> > > to reproduce it at all. Will keep trying. >> > >> > Sasha already gave a link to the syscalls sequence, so no rush. >> >> It'd be nice to get a more concise reproducer, his list had a little of everything in there. > > I've so far failed to find any explanation for your swapops.h BUG; > but believe I have identified one cause for "Bad rss-counter"s. > > My hunch is that the swapops.h BUG is "nearby", but I just cannot > fit it together (the swapops.h BUG comes when rmap cannot find all > all the migration entries it inserted earlier: it's a very useful > BUG for validating rmap). > > Untested patch below: I can't quite say Reported-by, because it may > not even be one that you and Sasha have been seeing; but I'm hopeful, > remap_file_pages is in the list. > > Please give this a try, preferably on 3.14-rc or earlier: I've never > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > a lot more than most of us); but have seen them on mmotm/next, so > some other trigger is coming up there, I'll worry about that once > it reaches 3.15-rc. The patch fixed the "Bad rss-counter" errors I've been seeing both in 3.14-rc7 and -next. Thanks, Sasha -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-f50.google.com (mail-qg0-f50.google.com [209.85.192.50]) by kanga.kvack.org (Postfix) with ESMTP id 03AE26B0138 for ; Tue, 18 Mar 2014 22:06:14 -0400 (EDT) Received: by mail-qg0-f50.google.com with SMTP id q108so23717876qgd.9 for ; Tue, 18 Mar 2014 19:06:14 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id gq5si10049726qab.3.2014.03.18.19.06.13 for ; Tue, 18 Mar 2014 19:06:14 -0700 (PDT) Date: Tue, 18 Mar 2014 22:06:02 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319020602.GA29787@redhat.com> References: <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5328F3B4.1080208@oracle.com> Sender: owner-linux-mm@kvack.org List-ID: To: Sasha Levin Cc: Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote: > > Untested patch below: I can't quite say Reported-by, because it may > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > remap_file_pages is in the list. > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > a lot more than most of us); but have seen them on mmotm/next, so > > some other trigger is coming up there, I'll worry about that once > > it reaches 3.15-rc. > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > 3.14-rc7 and -next. It's looking good here too so far. I'll leave it running overnight to be sure. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f41.google.com (mail-pb0-f41.google.com [209.85.160.41]) by kanga.kvack.org (Postfix) with ESMTP id 9FA1A6B013A for ; Tue, 18 Mar 2014 22:07:03 -0400 (EDT) Received: by mail-pb0-f41.google.com with SMTP id jt11so8151905pbb.14 for ; Tue, 18 Mar 2014 19:07:03 -0700 (PDT) Received: from mail-pb0-x22e.google.com (mail-pb0-x22e.google.com [2607:f8b0:400e:c01::22e]) by mx.google.com with ESMTPS id tm9si13195541pab.305.2014.03.18.19.07.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 19:07:02 -0700 (PDT) Received: by mail-pb0-f46.google.com with SMTP id rq2so8106312pbb.5 for ; Tue, 18 Mar 2014 19:07:02 -0700 (PDT) Date: Tue, 18 Mar 2014 19:06:02 -0700 (PDT) From: Hugh Dickins Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Linus Torvalds Cc: Hugh Dickins , Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, 18 Mar 2014, Linus Torvalds wrote: > On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins wrote: > > > > And yes, it is possible (though very unusual) to find an anon page or > > swap entry in a VM_SHARED nonlinear mapping: coming from that horrid > > get_user_pages(write, force) case which COWs even in a shared mapping. > > Hmm. Maybe we could just disallow that forced case. > > It *used* to be a trivial "we can just do a COW", but that was back > when the VM was much simpler and we had no rmap's etc. So "that horrid > case" used to be a simple hack that wasn't painful. But I suspect we > could very easily just fail it instead of forcing a COW, if that would > make it simpler for the VM code. I'd love that, if we can get away with it now: depends very much on whether we then turn out to break userspace or not. If I remember correctly, it's been that way since early days, in case ptrace were used to put a breakpoint into a MAP_SHARED mapping of an executable: to prevent that modification from reaching the file, if the file happened to be opened O_RDWR. Usually it's not open for writing, and mapped MAP_PRIVATE anyway. That is still something worth protecting against, I presume; but I'd much rather do it by failing the awkward case, than by perverting the VM to break its own rules. If I'm not mistaken, Konstantin (who happens to be already on this Cc list) had a patch (that I hated) to complicate things, to fix up some of the inconsistencies arising from this very odd and overlooked corner-case. I think he'd prefer this simplification to his patch too. I'll look into it further, but not in haste. Hugh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f54.google.com (mail-qa0-f54.google.com [209.85.216.54]) by kanga.kvack.org (Postfix) with ESMTP id 95BA36B013C for ; Tue, 18 Mar 2014 22:11:42 -0400 (EDT) Received: by mail-qa0-f54.google.com with SMTP id w8so7818418qac.41 for ; Tue, 18 Mar 2014 19:11:42 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id i72si10950875qge.49.2014.03.18.19.11.41 for ; Tue, 18 Mar 2014 19:11:42 -0700 (PDT) Date: Tue, 18 Mar 2014 22:11:31 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319021131.GA30018@redhat.com> References: <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140319020602.GA29787@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Sasha Levin , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote: > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote: > > > > Untested patch below: I can't quite say Reported-by, because it may > > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > > remap_file_pages is in the list. > > > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > > a lot more than most of us); but have seen them on mmotm/next, so > > > some other trigger is coming up there, I'll worry about that once > > > it reaches 3.15-rc. > > > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > > 3.14-rc7 and -next. > > It's looking good here too so far. I'll leave it running overnight to be sure. Of course, that isn't going to happen. Immediately after posting this, I hit the swapops bug. Patch does seem to have cured the bad rss counters though. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) by kanga.kvack.org (Postfix) with ESMTP id E03EC6B013D for ; Tue, 18 Mar 2014 22:13:09 -0400 (EDT) Received: by mail-pd0-f177.google.com with SMTP id y10so7938258pdj.36 for ; Tue, 18 Mar 2014 19:13:09 -0700 (PDT) Received: from mail-pb0-x235.google.com (mail-pb0-x235.google.com [2607:f8b0:400e:c01::235]) by mx.google.com with ESMTPS id tv5si10861354pbc.89.2014.03.18.19.13.08 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 19:13:08 -0700 (PDT) Received: by mail-pb0-f53.google.com with SMTP id rp16so8174367pbb.12 for ; Tue, 18 Mar 2014 19:13:08 -0700 (PDT) Date: Tue, 18 Mar 2014 19:12:14 -0700 (PDT) From: Hugh Dickins Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <5328F3B4.1080208@oracle.com> Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Sasha Levin Cc: Hugh Dickins , Dave Jones , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, 18 Mar 2014, Sasha Levin wrote: > On 03/18/2014 08:38 PM, Hugh Dickins wrote: > > On Tue, 11 Mar 2014, Dave Jones wrote: > > > On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: > > > > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > > > > > > > > > Dave, iirc trinity can write log file pointing which exactly > > > syscall sequence > > > > > > was passed, right? Share it too please. > > > > > > > > > > Hm, I may have been mistaken, and the damage was done by a previous > > > run. > > > > > I went from being able to reproduce it almost instantly to now not > > > being able > > > > > to reproduce it at all. Will keep trying. > > > > > > > > Sasha already gave a link to the syscalls sequence, so no rush. > > > > > > It'd be nice to get a more concise reproducer, his list had a little of > > > everything in there. > > > > I've so far failed to find any explanation for your swapops.h BUG; > > but believe I have identified one cause for "Bad rss-counter"s. > > > > My hunch is that the swapops.h BUG is "nearby", but I just cannot > > fit it together (the swapops.h BUG comes when rmap cannot find all > > all the migration entries it inserted earlier: it's a very useful > > BUG for validating rmap). > > > > Untested patch below: I can't quite say Reported-by, because it may > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > remap_file_pages is in the list. > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > a lot more than most of us); but have seen them on mmotm/next, so > > some other trigger is coming up there, I'll worry about that once > > it reaches 3.15-rc. > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > 3.14-rc7 and -next. Great, thanks a lot, Sasha. I was afraid that you'd hit those swapops BUGs, which seemed perhaps to be paired with these; but glad to hear a positive. Let's see how Dave fares. (I've not forgotten shmem fallocate, by the way, but those probably aren't as high on my agenda as you'd like.) Hugh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f45.google.com (mail-pb0-f45.google.com [209.85.160.45]) by kanga.kvack.org (Postfix) with ESMTP id 9E9C26B0140 for ; Tue, 18 Mar 2014 22:20:13 -0400 (EDT) Received: by mail-pb0-f45.google.com with SMTP id uo5so8187739pbc.32 for ; Tue, 18 Mar 2014 19:20:13 -0700 (PDT) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [2607:f8b0:400e:c02::233]) by mx.google.com with ESMTPS id nd6si3619056pbc.354.2014.03.18.19.20.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 19:20:12 -0700 (PDT) Received: by mail-pd0-f179.google.com with SMTP id w10so7998184pde.10 for ; Tue, 18 Mar 2014 19:20:12 -0700 (PDT) Date: Tue, 18 Mar 2014 19:19:09 -0700 (PDT) From: Hugh Dickins Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <20140319021131.GA30018@redhat.com> Message-ID: References: <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Sasha Levin , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, 18 Mar 2014, Dave Jones wrote: > On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote: > > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote: > > > > > > Untested patch below: I can't quite say Reported-by, because it may > > > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > > > remap_file_pages is in the list. > > > > > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > > > a lot more than most of us); but have seen them on mmotm/next, so > > > > some other trigger is coming up there, I'll worry about that once > > > > it reaches 3.15-rc. > > > > > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > > > 3.14-rc7 and -next. > > > > It's looking good here too so far. I'll leave it running overnight to be sure. > > Of course, that isn't going to happen. Immediately after posting this, I hit the > swapops bug. Patch does seem to have cured the bad rss counters though. Another positive on the rss counters, great, thanks Dave. That encourages me to think again on the swapops BUG, but no promises. Hugh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vc0-f173.google.com (mail-vc0-f173.google.com [209.85.220.173]) by kanga.kvack.org (Postfix) with ESMTP id 202EC6B0144 for ; Tue, 18 Mar 2014 22:24:14 -0400 (EDT) Received: by mail-vc0-f173.google.com with SMTP id il7so8480460vcb.32 for ; Tue, 18 Mar 2014 19:24:13 -0700 (PDT) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [2607:f8b0:400c:c01::22f]) by mx.google.com with ESMTPS id i3si7187520vcp.52.2014.03.18.19.24.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 19:24:13 -0700 (PDT) Received: by mail-ve0-f175.google.com with SMTP id oz11so8056105veb.20 for ; Tue, 18 Mar 2014 19:24:13 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> Date: Tue, 18 Mar 2014 19:24:13 -0700 Message-ID: Subject: Re: bad rss-counter message in 3.14rc5 From: Linus Torvalds Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins wrote: > > I'd love that, if we can get away with it now: depends very > much on whether we then turn out to break userspace or not. Right. I suspect we can, though, but it's one of those "we can try it and see". Remind me early in the 3.15 merge window, and we can just turn the "force" case into an error case and see if anybody hollers. > If I remember correctly, it's been that way since early days, > in case ptrace were used to put a breakpoint into a MAP_SHARED > mapping of an executable: to prevent that modification from > reaching the file, if the file happened to be opened O_RDWR. > Usually it's not open for writing, and mapped MAP_PRIVATE anyway. Yes, it's been that way since the very beginning, I think it goes back pretty much as far as MAP_SHARED does. We used to play lots of games wrt MAP_SHARED - in fact I think we used to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for the longest time there was no "true" writable MAP_SHARED at all, but we did have a coherent MAP_PRIVATE and something like the indexer for nntpd wanted a read-only shared mapping of the nntp spool or something like that. I forget the details, it's a _loong_ time ago. So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used to make a lot more sense in that kind of situation, when MAP_SHARED vs MAP_PRIVATE was much less of a black-and-white thing. I really suspect nobody cares wrt ptrace, especially since presumably other systems haven't had those kinds of games (although who knows - HP-UX in particular had some of the shittiest mmap() implementations on the planet - it made even the original Linux mmap hacks look like a thing of pure beauty in comparison). Linus -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f45.google.com (mail-pb0-f45.google.com [209.85.160.45]) by kanga.kvack.org (Postfix) with ESMTP id D9F326B0146 for ; Tue, 18 Mar 2014 22:38:02 -0400 (EDT) Received: by mail-pb0-f45.google.com with SMTP id uo5so8183533pbc.4 for ; Tue, 18 Mar 2014 19:38:02 -0700 (PDT) Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [2607:f8b0:400e:c03::22e]) by mx.google.com with ESMTPS id sf3si8403671pac.42.2014.03.18.19.38.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 19:38:01 -0700 (PDT) Received: by mail-pa0-f46.google.com with SMTP id kp14so8198728pab.19 for ; Tue, 18 Mar 2014 19:38:01 -0700 (PDT) Date: Tue, 18 Mar 2014 19:37:01 -0700 (PDT) From: Hugh Dickins Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Linus Torvalds Cc: Hugh Dickins , Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, 18 Mar 2014, Linus Torvalds wrote: > On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins wrote: > > > > I'd love that, if we can get away with it now: depends very > > much on whether we then turn out to break userspace or not. > > Right. I suspect we can, though, but it's one of those "we can try it > and see". Remind me early in the 3.15 merge window, and we can just > turn the "force" case into an error case and see if anybody hollers. Super, I'll do that, thanks. For 3.15, and probably 3.16 too, we should keep in place whatever partial accommodations we have for the case (such as allowing for anon and swap in fremap's zap_pte), in case we do need to revert; but clean those away later on. (Not many, I think: it was mainly a guilty secret that VM accounting didn't really hold together.) > > > If I remember correctly, it's been that way since early days, > > in case ptrace were used to put a breakpoint into a MAP_SHARED > > mapping of an executable: to prevent that modification from > > reaching the file, if the file happened to be opened O_RDWR. > > Usually it's not open for writing, and mapped MAP_PRIVATE anyway. > > Yes, it's been that way since the very beginning, I think it goes back > pretty much as far as MAP_SHARED does. > > We used to play lots of games wrt MAP_SHARED - in fact I think we used > to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for > the longest time there was no "true" writable MAP_SHARED at all, but > we did have a coherent MAP_PRIVATE and something like the indexer for > nntpd wanted a read-only shared mapping of the nntp spool or something > like that. I forget the details, it's a _loong_ time ago. > > So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used > to make a lot more sense in that kind of situation, when MAP_SHARED vs > MAP_PRIVATE was much less of a black-and-white thing. > > I really suspect nobody cares wrt ptrace, especially since presumably > other systems haven't had those kinds of games (although who knows - > HP-UX in particular had some of the shittiest mmap() implementations > on the planet - it made even the original Linux mmap hacks look like a > thing of pure beauty in comparison). :) That fits with what I heard of HP-UX mmap, but I never had the pleasure of dealing with it. Hugh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f45.google.com (mail-pa0-f45.google.com [209.85.220.45]) by kanga.kvack.org (Postfix) with ESMTP id C86996B0147 for ; Tue, 18 Mar 2014 22:42:20 -0400 (EDT) Received: by mail-pa0-f45.google.com with SMTP id kl14so8246441pab.32 for ; Tue, 18 Mar 2014 19:42:20 -0700 (PDT) Received: from userp1040.oracle.com (userp1040.oracle.com. [156.151.31.81]) by mx.google.com with ESMTPS id my2si12055298pbc.145.2014.03.18.19.42.19 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 18 Mar 2014 19:42:19 -0700 (PDT) Message-ID: <53290403.5030809@oracle.com> Date: Tue, 18 Mar 2014 22:42:11 -0400 From: Sasha Levin MIME-Version: 1.0 Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Dave Jones , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On 03/18/2014 10:12 PM, Hugh Dickins wrote: > On Tue, 18 Mar 2014, Sasha Levin wrote: >> On 03/18/2014 08:38 PM, Hugh Dickins wrote: >>> On Tue, 11 Mar 2014, Dave Jones wrote: >>>> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: >>>> > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: >>>> > > > >>>> > > > Dave, iirc trinity can write log file pointing which exactly >>>> syscall sequence >>>> > > > was passed, right? Share it too please. >>>> > > >>>> > > Hm, I may have been mistaken, and the damage was done by a previous >>>> run. >>>> > > I went from being able to reproduce it almost instantly to now not >>>> being able >>>> > > to reproduce it at all. Will keep trying. >>>> > >>>> > Sasha already gave a link to the syscalls sequence, so no rush. >>>> >>>> It'd be nice to get a more concise reproducer, his list had a little of >>>> everything in there. >>> >>> I've so far failed to find any explanation for your swapops.h BUG; >>> but believe I have identified one cause for "Bad rss-counter"s. >>> >>> My hunch is that the swapops.h BUG is "nearby", but I just cannot >>> fit it together (the swapops.h BUG comes when rmap cannot find all >>> all the migration entries it inserted earlier: it's a very useful >>> BUG for validating rmap). >>> >>> Untested patch below: I can't quite say Reported-by, because it may >>> not even be one that you and Sasha have been seeing; but I'm hopeful, >>> remap_file_pages is in the list. >>> >>> Please give this a try, preferably on 3.14-rc or earlier: I've never >>> seen "Bad rss-counter"s there myself (trinity uses remap_file_pages >>> a lot more than most of us); but have seen them on mmotm/next, so >>> some other trigger is coming up there, I'll worry about that once >>> it reaches 3.15-rc. >> >> The patch fixed the "Bad rss-counter" errors I've been seeing both in >> 3.14-rc7 and -next. > > Great, thanks a lot, Sasha. I was afraid that you'd hit those swapops > BUGs, which seemed perhaps to be paired with these; but glad to hear > a positive. Let's see how Dave fares. (I've not forgotten shmem > fallocate, by the way, but those probably aren't as high on my agenda > as you'd like.) I do hit the swapops issue a lot, I didn't think that your patch was supposed to fix that so I didn't mention it. Thanks for keeping shmem in mind, I've removed shmem from testing for now but I agree, it's not one of the more important issues to be taken care of. Thanks, Sasha -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ve0-f174.google.com (mail-ve0-f174.google.com [209.85.128.174]) by kanga.kvack.org (Postfix) with ESMTP id 5D18D6B014C for ; Tue, 18 Mar 2014 22:57:01 -0400 (EDT) Received: by mail-ve0-f174.google.com with SMTP id oz11so7947823veb.5 for ; Tue, 18 Mar 2014 19:57:01 -0700 (PDT) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [2607:f8b0:400c:c01::22f]) by mx.google.com with ESMTPS id w5si7200661vcl.141.2014.03.18.19.57.00 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Mar 2014 19:57:00 -0700 (PDT) Received: by mail-ve0-f175.google.com with SMTP id oz11so8135869veb.34 for ; Tue, 18 Mar 2014 19:57:00 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> Date: Tue, 18 Mar 2014 19:57:00 -0700 Message-ID: Subject: Re: bad rss-counter message in 3.14rc5 From: Linus Torvalds Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 18, 2014 at 7:37 PM, Hugh Dickins wrote: > > For 3.15, and probably 3.16 too, we should keep in place whatever > partial accommodations we have for the case (such as allowing for > anon and swap in fremap's zap_pte), in case we do need to revert; > but clean those away later on. (Not many, I think: it was mainly > a guilty secret that VM accounting didn't really hold together.) Absolutely. See if it works to just stop doing that special COW, and then later on, if we have decided "nobody even noticed", we can remove the hacks we have to support the fact that shared mappings sometimes have anon pages in them. > :) That fits with what I heard of HP-UX mmap, > but I never had the pleasure of dealing with it. They had purely virtually indexed caches, making coherency "interesting". Together with a VM based on some really old BSD VM code that everybody else had thrown out, and that didn't allow you to unmap things partially etc. So HPUX mmap really didn't work, not even for non-shared mmap's. I think they fixed the interfaces in HP-UX 11. But not being coherent meant that the shared mappings tended to still have trouble. nntp largely died, but was replaced with the cyrus imapd that played similar games. At least out mmap was always coherent. Even in MAP_PRIVATE, and with regards to both write() system calls and other mmap PROT_WRITE users. Except when we had bugs. Shared mmap really isn't very simple to get right. Linus -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-we0-f171.google.com (mail-we0-f171.google.com [74.125.82.171]) by kanga.kvack.org (Postfix) with ESMTP id BD38D6B015A for ; Wed, 19 Mar 2014 07:04:40 -0400 (EDT) Received: by mail-we0-f171.google.com with SMTP id t61so6799301wes.2 for ; Wed, 19 Mar 2014 04:04:40 -0700 (PDT) Received: from mx2.suse.de (cantor2.suse.de. [195.135.220.15]) by mx.google.com with ESMTPS id db6si9587047wib.25.2014.03.19.04.04.38 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 19 Mar 2014 04:04:39 -0700 (PDT) Date: Wed, 19 Mar 2014 12:04:36 +0100 From: Jan Kara Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319110436.GF26358@quack.suse.cz> References: <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Linus Torvalds , Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue 18-03-14 19:37:01, Hugh Dickins wrote: > On Tue, 18 Mar 2014, Linus Torvalds wrote: > > On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins wrote: > > > > > > I'd love that, if we can get away with it now: depends very > > > much on whether we then turn out to break userspace or not. > > > > Right. I suspect we can, though, but it's one of those "we can try it > > and see". Remind me early in the 3.15 merge window, and we can just > > turn the "force" case into an error case and see if anybody hollers. > > Super, I'll do that, thanks. > > For 3.15, and probably 3.16 too, we should keep in place whatever > partial accommodations we have for the case (such as allowing for > anon and swap in fremap's zap_pte), in case we do need to revert; > but clean those away later on. (Not many, I think: it was mainly > a guilty secret that VM accounting didn't really hold together.) Different drivers actually use the 'force' argument of get_user_pages() a lot on userspace provided buffers (AFAIU because they want to tell the kernel HW is going to write to that memory so they want to prepare for it). It is hard to imagine someone will use this for MAP_SHARED pages (or what that would be supposed to achieve) but sometimes userspace is surprisingly inventive... Just something to be aware of... Honza -- Jan Kara SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-f180.google.com (mail-lb0-f180.google.com [209.85.217.180]) by kanga.kvack.org (Postfix) with ESMTP id F3CC16B0162 for ; Wed, 19 Mar 2014 08:04:27 -0400 (EDT) Received: by mail-lb0-f180.google.com with SMTP id 10so5879693lbg.11 for ; Wed, 19 Mar 2014 05:04:26 -0700 (PDT) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [2a00:1450:4010:c03::236]) by mx.google.com with ESMTPS id be6si10608246lbc.186.2014.03.19.05.04.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 19 Mar 2014 05:04:26 -0700 (PDT) Received: by mail-la0-f54.google.com with SMTP id mc6so5869127lab.13 for ; Wed, 19 Mar 2014 05:04:25 -0700 (PDT) Date: Wed, 19 Mar 2014 16:04:24 +0400 From: Cyrill Gorcunov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319120424.GD1728@moon> References: <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Dave Jones , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 18, 2014 at 05:38:38PM -0700, Hugh Dickins wrote: > > (Cyrill, entirely unrelated, but in preparing this patch I noticed > your soft_dirty work in install_file_pte(): which looked good at > first, until I realized that it's propagating the soft_dirty of a > pte it's about to zap completely, to the unrelated entry it's about > to insert in its place. Which seems very odd to me.) > Thanks a lot Hugh for pointing! I'll revisit all file-softdirty cases. (btw, I've grabbed Dave's config to run trinity and somehow help in testing and attempt to figure out what causes it but didn't yet find hardware node to run, hopefully i'll get a spare machine for testing in a couple of days). -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-f179.google.com (mail-qc0-f179.google.com [209.85.216.179]) by kanga.kvack.org (Postfix) with ESMTP id C60666B0167 for ; Wed, 19 Mar 2014 10:52:11 -0400 (EDT) Received: by mail-qc0-f179.google.com with SMTP id m20so9731420qcx.24 for ; Wed, 19 Mar 2014 07:52:11 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id c8si5711170qco.32.2014.03.19.07.52.10 for ; Wed, 19 Mar 2014 07:52:11 -0700 (PDT) Date: Wed, 19 Mar 2014 10:52:00 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319145200.GA4608@redhat.com> References: <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote: > Another positive on the rss counters, great, thanks Dave. > That encourages me to think again on the swapops BUG, but no promises. So while I slept I ran a test kernel with that swapops BUG replaced with a printk. I'm not sure of the validity of this, given the state of the kernel afterwards is somewhat suspect, but I did see in the logs this morning.. [18728.075153] migration_entry_to_page BUG hit [18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted) [18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted) This might be collateral damage from the swapops thing, I guess we won't know until that gets fixed, but I thought I'd mention that we might still have a problem here. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f176.google.com (mail-pd0-f176.google.com [209.85.192.176]) by kanga.kvack.org (Postfix) with ESMTP id C67296B018E for ; Thu, 20 Mar 2014 01:01:38 -0400 (EDT) Received: by mail-pd0-f176.google.com with SMTP id r10so383614pdi.35 for ; Wed, 19 Mar 2014 22:01:38 -0700 (PDT) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [2607:f8b0:400e:c02::22f]) by mx.google.com with ESMTPS id s3si528200pbo.217.2014.03.19.22.01.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 19 Mar 2014 22:01:31 -0700 (PDT) Received: by mail-pd0-f175.google.com with SMTP id x10so386088pdj.20 for ; Wed, 19 Mar 2014 22:01:30 -0700 (PDT) Date: Wed, 19 Mar 2014 22:00:29 -0700 (PDT) From: Hugh Dickins Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <20140319145200.GA4608@redhat.com> Message-ID: References: <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones Cc: Hugh Dickins , Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Wed, 19 Mar 2014, Dave Jones wrote: > On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote: > > > Another positive on the rss counters, great, thanks Dave. > > That encourages me to think again on the swapops BUG, but no promises. > > So while I slept I ran a test kernel with that swapops BUG replaced with a printk. > I'm not sure of the validity of this, given the state of the kernel afterwards > is somewhat suspect, but I did see in the logs this morning.. > > [18728.075153] migration_entry_to_page BUG hit > [18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted) > [18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted) > > This might be collateral damage from the swapops thing, I guess we won't know until > that gets fixed, but I thought I'd mention that we might still have a problem here. Yes, those Bad rss-counters could well be collateral damage from the swapops BUG. To which I believe I now have the answer: again untested, but please give this a try... (It's worth saying, by the way, that these bugs are not a consequence of recent changes at all, they've been there for ages; but trinity has just got better at taunting remap_file_pages and the rest of mm...) [PATCH] mm: fix swapops.h:131 bug if remap_file_pages raced migration Add remove_linear_migration_ptes_from_nonlinear(), to fix an interesting little include/linux/swapops.h:131 BUG_ON(!PageLocked) found by trinity: indicating that remove_migration_ptes() failed to find one of the migration entries that was temporarily inserted. The problem comes from remap_file_pages()'s switch from vma_interval_tree (good for inserting the migration entry) to i_mmap_nonlinear list (no good for locating it again); but can only be a problem if the remap_file_pages() range does not cover the whole of the vma (zap_pte() clears the range). remove_migration_ptes() needs a file_nonlinear method to go down the i_mmap_nonlinear list, applying linear location to look for migration entries in those vmas too, just in case there was this race. The file_nonlinear method does need rmap_walk_control.arg to do this; but it never needed vma passed in - vma comes from its own iteration. Signed-off-by: Hugh Dickins --- include/linux/rmap.h | 3 +-- mm/migrate.c | 32 ++++++++++++++++++++++++++++++++ mm/rmap.c | 5 +++-- 3 files changed, 36 insertions(+), 4 deletions(-) --- 3.14-rc7/include/linux/rmap.h 2014-02-02 18:49:07.429302104 -0800 +++ linux/include/linux/rmap.h 2014-03-19 20:12:27.056451541 -0700 @@ -250,8 +250,7 @@ struct rmap_walk_control { int (*rmap_one)(struct page *page, struct vm_area_struct *vma, unsigned long addr, void *arg); int (*done)(struct page *page); - int (*file_nonlinear)(struct page *, struct address_space *, - struct vm_area_struct *vma); + int (*file_nonlinear)(struct page *, struct address_space *, void *arg); struct anon_vma *(*anon_lock)(struct page *page); bool (*invalid_vma)(struct vm_area_struct *vma, void *arg); }; --- 3.14-rc7/mm/migrate.c 2014-03-16 19:24:19.635512576 -0700 +++ linux/mm/migrate.c 2014-03-19 21:06:02.704527965 -0700 @@ -178,6 +178,37 @@ out: } /* + * Congratulations to trinity for discovering this bug. + * mm/fremap.c's remap_file_pages() accepts any range within a single vma to + * convert that vma to VM_NONLINEAR; and generic_file_remap_pages() will then + * replace the specified range by file ptes throughout (maybe populated after). + * If page migration finds a page within that range, while it's still located + * by vma_interval_tree rather than lost to i_mmap_nonlinear list, no problem: + * zap_pte() clears the temporary migration entry before mmap_sem is dropped. + * But if the migrating page is in a part of the vma outside the range to be + * remapped, then it will not be cleared, and remove_migration_ptes() needs to + * deal with it. Fortunately, this part of the vma is of course still linear, + * so we just need to use linear location on the nonlinear list. + */ +static int remove_linear_migration_ptes_from_nonlinear(struct page *page, + struct address_space *mapping, void *arg) +{ + struct vm_area_struct *vma; + /* hugetlbfs does not support remap_pages, so no huge pgoff worries */ + pgoff_t pgoff = page->index << (PAGE_CACHE_SHIFT - PAGE_SHIFT); + unsigned long addr; + + list_for_each_entry(vma, + &mapping->i_mmap_nonlinear, shared.nonlinear) { + + addr = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT); + if (addr >= vma->vm_start && addr < vma->vm_end) + remove_migration_pte(page, vma, addr, arg); + } + return SWAP_AGAIN; +} + +/* * Get rid of all migration entries and replace them by * references to the indicated page. */ @@ -186,6 +217,7 @@ static void remove_migration_ptes(struct struct rmap_walk_control rwc = { .rmap_one = remove_migration_pte, .arg = old, + .file_nonlinear = remove_linear_migration_ptes_from_nonlinear, }; rmap_walk(new, &rwc); --- 3.14-rc7/mm/rmap.c 2014-02-02 18:49:07.929302115 -0800 +++ linux/mm/rmap.c 2014-03-19 20:16:03.552456686 -0700 @@ -1360,8 +1360,9 @@ static int try_to_unmap_cluster(unsigned } static int try_to_unmap_nonlinear(struct page *page, - struct address_space *mapping, struct vm_area_struct *vma) + struct address_space *mapping, void *arg) { + struct vm_area_struct *vma; int ret = SWAP_AGAIN; unsigned long cursor; unsigned long max_nl_cursor = 0; @@ -1663,7 +1664,7 @@ static int rmap_walk_file(struct page *p if (list_empty(&mapping->i_mmap_nonlinear)) goto done; - ret = rwc->file_nonlinear(page, mapping, vma); + ret = rwc->file_nonlinear(page, mapping, rwc->arg); done: mutex_unlock(&mapping->i_mmap_mutex); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f51.google.com (mail-qa0-f51.google.com [209.85.216.51]) by kanga.kvack.org (Postfix) with ESMTP id BB2AA6B0202 for ; Thu, 20 Mar 2014 09:51:49 -0400 (EDT) Received: by mail-qa0-f51.google.com with SMTP id j7so870823qaq.10 for ; Thu, 20 Mar 2014 06:51:49 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTP id 4si775800qat.140.2014.03.20.06.51.48 for ; Thu, 20 Mar 2014 06:51:49 -0700 (PDT) Date: Thu, 20 Mar 2014 09:51:37 -0400 From: Dave Jones Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140320135137.GA2263@redhat.com> References: <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Hugh Dickins Cc: Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote: > > This might be collateral damage from the swapops thing, I guess we won't know until > > that gets fixed, but I thought I'd mention that we might still have a problem here. > > Yes, those Bad rss-counters could well be collateral damage from the > swapops BUG. To which I believe I now have the answer: again untested, > but please give this a try... This survived an overnight run. No swapops bug, and no bad RSS. Good job :) > (It's worth saying, by the way, that these bugs are not a consequence > of recent changes at all, they've been there for ages; but trinity has > just got better at taunting remap_file_pages and the rest of mm...) Indeed. I hope to lift the covers on more stuff like this (and hopefully get it done in a more reproducable manner). A lot of the stuff trinity is doing with VM syscalls is still very naive. Dave -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) by kanga.kvack.org (Postfix) with ESMTP id 261006B0206 for ; Thu, 20 Mar 2014 10:19:32 -0400 (EDT) Received: by mail-pa0-f53.google.com with SMTP id ld10so1009763pab.12 for ; Thu, 20 Mar 2014 07:19:31 -0700 (PDT) Received: from aserp1040.oracle.com (aserp1040.oracle.com. [141.146.126.69]) by mx.google.com with ESMTPS id yo5si1596982pab.292.2014.03.20.07.19.30 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 20 Mar 2014 07:19:30 -0700 (PDT) Message-ID: <532AF8E8.8030101@oracle.com> Date: Thu, 20 Mar 2014 10:19:20 -0400 From: Sasha Levin MIME-Version: 1.0 Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> <20140320135137.GA2263@redhat.com> In-Reply-To: <20140320135137.GA2263@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Dave Jones , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On 03/20/2014 09:51 AM, Dave Jones wrote: > On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote: > > > > This might be collateral damage from the swapops thing, I guess we won't know until > > > that gets fixed, but I thought I'd mention that we might still have a problem here. > > > > Yes, those Bad rss-counters could well be collateral damage from the > > swapops BUG. To which I believe I now have the answer: again untested, > > but please give this a try... > > This survived an overnight run. No swapops bug, and no bad RSS. Good job:) Same here, swapops bug is gone! Thanks, Sasha -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f47.google.com (mail-pb0-f47.google.com [209.85.160.47]) by kanga.kvack.org (Postfix) with ESMTP id 03D836B0266 for ; Fri, 21 Mar 2014 00:47:58 -0400 (EDT) Received: by mail-pb0-f47.google.com with SMTP id up15so1910679pbc.34 for ; Thu, 20 Mar 2014 21:47:58 -0700 (PDT) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [2607:f8b0:400e:c03::22b]) by mx.google.com with ESMTPS id zt8si2809922pbc.316.2014.03.20.21.47.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 20 Mar 2014 21:47:54 -0700 (PDT) Received: by mail-pa0-f43.google.com with SMTP id bj1so1921515pad.30 for ; Thu, 20 Mar 2014 21:47:53 -0700 (PDT) Date: Thu, 20 Mar 2014 21:46:51 -0700 (PDT) From: Hugh Dickins Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <532AF8E8.8030101@oracle.com> Message-ID: References: <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> <20140320135137.GA2263@redhat.com> <532AF8E8.8030101@oracle.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Sasha Levin , Dave Jones Cc: Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov On Thu, 20 Mar 2014, Sasha Levin wrote: > On 03/20/2014 09:51 AM, Dave Jones wrote: > > On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote: > > > > > > This might be collateral damage from the swapops thing, I guess we > > won't know until > > > > that gets fixed, but I thought I'd mention that we might still have a > > problem here. > > > > > > Yes, those Bad rss-counters could well be collateral damage from the > > > swapops BUG. To which I believe I now have the answer: again untested, > > > but please give this a try... > > > > This survived an overnight run. No swapops bug, and no bad RSS. Good job:) > > Same here, swapops bug is gone! That was welcome news, thanks guys. I notice it has not (yet) magically appeared in Linus's public tree like the rss one did: so to be on the safe side, I'll just repost it now, with your Reported-and-tested-bys, otherwise unchanged. Hugh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756610AbaCERpX (ORCPT ); Wed, 5 Mar 2014 12:45:23 -0500 Received: from mx1.redhat.com ([209.132.183.28]:63024 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754298AbaCERpV (ORCPT ); Wed, 5 Mar 2014 12:45:21 -0500 Date: Wed, 5 Mar 2014 12:45:03 -0500 From: Dave Jones To: Linux Kernel Cc: linux-mm@kvack.org, Linus Torvalds , Andrew Morton Subject: bad rss-counter message in 3.14rc5 Message-ID: <20140305174503.GA16335@redhat.com> Mail-Followup-To: Dave Jones , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I just saw this on my box that's been running trinity.. [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) There's nothing else, no trace, nothing. Any ideas where to begin with this? Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755619AbaCER5h (ORCPT ); Wed, 5 Mar 2014 12:57:37 -0500 Received: from mx1.redhat.com ([209.132.183.28]:28221 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753029AbaCER5g (ORCPT ); Wed, 5 Mar 2014 12:57:36 -0500 Date: Wed, 5 Mar 2014 12:57:25 -0500 From: Dave Jones To: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140305175725.GB16335@redhat.com> Mail-Followup-To: Dave Jones , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton References: <20140305174503.GA16335@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140305174503.GA16335@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote: > I just saw this on my box that's been running trinity.. > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > There's nothing else, no trace, nothing. Any ideas where to begin with this? ah, on the serial console there was also this truncated warning.. [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) [48924.133273] ------------[ cut here ]------------ [48924.133391] kernel BUG at include/linux/swapops.h:131! Dave 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) 125 { 126 struct page *p = pfn_to_page(swp_offset(entry)); 127 /* 128 * Any use of migration entries may only occur while the 129 * corresponding page is locked 130 */ 131 BUG_ON(!PageLocked(p)); 132 return p; 133 } From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752078AbaCGAWY (ORCPT ); Thu, 6 Mar 2014 19:22:24 -0500 Received: from mx1.redhat.com ([209.132.183.28]:62192 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750919AbaCGAWW (ORCPT ); Thu, 6 Mar 2014 19:22:22 -0500 Date: Thu, 6 Mar 2014 19:22:10 -0500 From: Dave Jones To: Linux Kernel Cc: linux-mm@kvack.org, Linus Torvalds , Andrew Morton Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140307002210.GA26603@redhat.com> Mail-Followup-To: Dave Jones , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140305175725.GB16335@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote: > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote: > > I just saw this on my box that's been running trinity.. > > > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > > > There's nothing else, no trace, nothing. Any ideas where to begin with this? > > ah, on the serial console there was also this truncated warning.. > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > [48924.133273] ------------[ cut here ]------------ > [48924.133391] kernel BUG at include/linux/swapops.h:131! > > Dave > > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) > 125 { > 126 struct page *p = pfn_to_page(swp_offset(entry)); > 127 /* > 128 * Any use of migration entries may only occur while the > 129 * corresponding page is locked > 130 */ > 131 BUG_ON(!PageLocked(p)); > 132 return p; > 133 } I hit this again, this time a full trace made it over the serial console. This time there was no bad rss-counter message though. kernel BUG at include/linux/swapops.h:131! invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000 RIP: 0010:[] [] migration_entry_to_page.part.47+0x4/0x6 RSP: 0000:ffff88018911bae8 EFLAGS: 00010246 RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26 RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000 FS: 0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0 DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 Stack: ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98 Call Trace: [] unmap_single_vma+0x89e/0x8a0 [] unmap_vmas+0x49/0x90 [] exit_mmap+0xe5/0x1a0 [] mmput+0x73/0x110 [] do_exit+0x2a2/0xb50 [] ? __sigqueue_free.part.11+0x33/0x40 [] ? __dequeue_signal+0x13c/0x220 [] do_group_exit+0x4c/0xc0 [] get_signal_to_deliver+0x2d1/0x6d0 [] do_signal+0x57/0x9d0 [] ? __acct_update_integrals+0x8e/0x120 [] ? preempt_count_sub+0x6b/0xf0 [] ? _raw_spin_unlock+0x31/0x50 [] ? vtime_account_user+0x91/0xa0 [] ? context_tracking_user_exit+0x9b/0x100 [] do_notify_resume+0x71/0xc0 [] retint_signal+0x46/0x90 Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753794AbaCKCte (ORCPT ); Mon, 10 Mar 2014 22:49:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36792 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753584AbaCKCtS (ORCPT ); Mon, 10 Mar 2014 22:49:18 -0400 Date: Mon, 10 Mar 2014 22:49:06 -0400 From: Dave Jones To: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311024906.GA9191@redhat.com> Mail-Followup-To: Dave Jones , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Andrew Morton References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140307002210.GA26603@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 06, 2014 at 07:22:10PM -0500, Dave Jones wrote: > On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote: > > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote: > > > I just saw this on my box that's been running trinity.. > > > > > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > > > > > There's nothing else, no trace, nothing. Any ideas where to begin with this? > > > > ah, on the serial console there was also this truncated warning.. > > > > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted) > > [48924.133273] ------------[ cut here ]------------ > > [48924.133391] kernel BUG at include/linux/swapops.h:131! > > > > Dave > > > > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) > > 125 { > > 126 struct page *p = pfn_to_page(swp_offset(entry)); > > 127 /* > > 128 * Any use of migration entries may only occur while the > > 129 * corresponding page is locked > > 130 */ > > 131 BUG_ON(!PageLocked(p)); > > 132 return p; > > 133 } > > I hit this again, this time a full trace made it over the serial console. > This time there was no bad rss-counter message though. > > kernel BUG at include/linux/swapops.h:131! > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000 > RIP: 0010:[] [] migration_entry_to_page.part.47+0x4/0x6 > RSP: 0000:ffff88018911bae8 EFLAGS: 00010246 > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000 > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26 > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000 > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000 > FS: 0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0 > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000 > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 > Stack: > ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff > ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000 > 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98 > Call Trace: > [] unmap_single_vma+0x89e/0x8a0 > [] unmap_vmas+0x49/0x90 > [] exit_mmap+0xe5/0x1a0 > [] mmput+0x73/0x110 > [] do_exit+0x2a2/0xb50 > [] ? __sigqueue_free.part.11+0x33/0x40 > [] ? __dequeue_signal+0x13c/0x220 > [] do_group_exit+0x4c/0xc0 > [] get_signal_to_deliver+0x2d1/0x6d0 > [] do_signal+0x57/0x9d0 > [] ? __acct_update_integrals+0x8e/0x120 > [] ? preempt_count_sub+0x6b/0xf0 > [] ? _raw_spin_unlock+0x31/0x50 > [] ? vtime_account_user+0x91/0xa0 > [] ? context_tracking_user_exit+0x9b/0x100 > [] do_notify_resume+0x71/0xc0 > [] retint_signal+0x46/0x90 > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8 Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug.. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754219AbaCKDJR (ORCPT ); Mon, 10 Mar 2014 23:09:17 -0400 Received: from mail.linuxfoundation.org ([140.211.169.12]:37892 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753018AbaCKDJQ (ORCPT ); Mon, 10 Mar 2014 23:09:16 -0400 Date: Mon, 10 Mar 2014 20:13:40 -0700 From: Andrew Morton To: Dave Jones Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-Id: <20140310201340.81994295.akpm@linux-foundation.org> In-Reply-To: <20140311024906.GA9191@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.9; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 10 Mar 2014 22:49:06 -0400 Dave Jones wrote: > ... > > > > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry) > > > 125 { > > > 126 struct page *p = pfn_to_page(swp_offset(entry)); > > > 127 /* > > > 128 * Any use of migration entries may only occur while the > > > 129 * corresponding page is locked > > > 130 */ > > > 131 BUG_ON(!PageLocked(p)); > > > 132 return p; > > > 133 } > > > > I hit this again, this time a full trace made it over the serial console. > > This time there was no bad rss-counter message though. > > > > kernel BUG at include/linux/swapops.h:131! > > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC > > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore > > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 > > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000 > > RIP: 0010:[] [] migration_entry_to_page.part.47+0x4/0x6 > > RSP: 0000:ffff88018911bae8 EFLAGS: 00010246 > > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000 > > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26 > > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000 > > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c > > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000 > > FS: 0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000 > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0 > > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000 > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 > > Stack: > > ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff > > ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000 > > 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98 > > Call Trace: > > [] unmap_single_vma+0x89e/0x8a0 > > [] unmap_vmas+0x49/0x90 > > [] exit_mmap+0xe5/0x1a0 > > [] mmput+0x73/0x110 > > [] do_exit+0x2a2/0xb50 > > [] ? __sigqueue_free.part.11+0x33/0x40 > > [] ? __dequeue_signal+0x13c/0x220 > > [] do_group_exit+0x4c/0xc0 > > [] get_signal_to_deliver+0x2d1/0x6d0 > > [] do_signal+0x57/0x9d0 > > [] ? __acct_update_integrals+0x8e/0x120 > > [] ? preempt_count_sub+0x6b/0xf0 > > [] ? _raw_spin_unlock+0x31/0x50 > > [] ? vtime_account_user+0x91/0xa0 > > [] ? context_tracking_user_exit+0x9b/0x100 > > [] do_notify_resume+0x71/0xc0 > > [] retint_signal+0x46/0x90 > > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8 > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > while trying to reproduce a different bug.. Damn, I thought we'd fixed that but it seems not. Cc's added. Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()? From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754108AbaCKEmQ (ORCPT ); Tue, 11 Mar 2014 00:42:16 -0400 Received: from mail.linuxfoundation.org ([140.211.169.12]:38419 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753639AbaCKEls (ORCPT ); Tue, 11 Mar 2014 00:41:48 -0400 Date: Mon, 10 Mar 2014 21:46:12 -0700 From: Andrew Morton To: Dave Jones , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-Id: <20140310214612.3b4de36a.akpm@linux-foundation.org> In-Reply-To: <20140310201340.81994295.akpm@linux-foundation.org> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.9; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > while trying to reproduce a different bug.. > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > Guys, what stops the migration target page from coming unlocked in > parallel with zap_pte_range()'s call to migration_entry_to_page()? page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl. I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753695AbaCKEu2 (ORCPT ); Tue, 11 Mar 2014 00:50:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:3704 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752337AbaCKEu1 (ORCPT ); Tue, 11 Mar 2014 00:50:27 -0400 Date: Tue, 11 Mar 2014 00:50:05 -0400 From: Dave Jones To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311045005.GA12551@redhat.com> Mail-Followup-To: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310214612.3b4de36a.akpm@linux-foundation.org> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > while trying to reproduce a different bug.. > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > Guys, what stops the migration target page from coming unlocked in > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > and page_locked() happen under ptl. > > I don't see any holes in regular migration. Do you know if this is > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? I'll give it an overnight run and let you know tomorrow. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753447AbaCKEvY (ORCPT ); Tue, 11 Mar 2014 00:51:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:5704 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751257AbaCKEvX (ORCPT ); Tue, 11 Mar 2014 00:51:23 -0400 Date: Tue, 11 Mar 2014 00:51:09 -0400 From: Dave Jones To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311045109.GB12551@redhat.com> Mail-Followup-To: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310214612.3b4de36a.akpm@linux-foundation.org> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > while trying to reproduce a different bug.. > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > Guys, what stops the migration target page from coming unlocked in > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > and page_locked() happen under ptl. > > I don't see any holes in regular migration. Do you know if this is > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753521AbaCKE5f (ORCPT ); Tue, 11 Mar 2014 00:57:35 -0400 Received: from mail.linuxfoundation.org ([140.211.169.12]:38544 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752504AbaCKE5d (ORCPT ); Tue, 11 Mar 2014 00:57:33 -0400 Date: Mon, 10 Mar 2014 22:01:58 -0700 From: Andrew Morton To: Dave Jones Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-Id: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> In-Reply-To: <20140311045109.GB12551@redhat.com> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.9; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > while trying to reproduce a different bug.. > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > Guys, what stops the migration target page from coming unlocked in > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > and page_locked() happen under ptl. > > > > I don't see any holes in regular migration. Do you know if this is > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. There probably isn't much point unless trinity is using sys_move_pages(). Is it? If so it would be interesting to disable trinity's move_pages calls and see if it still fails. Grasping at straws here, trying to reduce the amount of code to look at :( From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754477AbaCKFIL (ORCPT ); Tue, 11 Mar 2014 01:08:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:26408 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752342AbaCKFII (ORCPT ); Tue, 11 Mar 2014 01:08:08 -0400 Date: Tue, 11 Mar 2014 01:07:52 -0400 From: Dave Jones To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311050752.GA14329@redhat.com> Mail-Followup-To: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Wanpeng Li , Bob Liu , Konstantin Khlebnikov References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote: > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > > while trying to reproduce a different bug.. > > > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > > > Guys, what stops the migration target page from coming unlocked in > > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > > and page_locked() happen under ptl. > > > > > > I don't see any holes in regular migration. Do you know if this is > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > There probably isn't much point unless trinity is using > sys_move_pages(). Is it? Trinity will do every syscall an arch has. In the test case I have so far, I've narrowed it down to the vm group of syscalls (so running with '-g vm' will do anything that I deemed 'vm'. Including.. sys_move_pages) I'll try to narrow it down further tomorrow. > If so it would be interesting to disable > trinity's move_pages calls and see if it still fails. Ok, I'll try that first. > Grasping at straws here, trying to reduce the amount of code to look at :( *nod*, it's not helped by the fact that the trace happens at process exit time which could be considerably later after the syscall that buggers everything up has happened. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754262AbaCKFbH (ORCPT ); Tue, 11 Mar 2014 01:31:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50156 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753000AbaCKFbE (ORCPT ); Tue, 11 Mar 2014 01:31:04 -0400 Date: Tue, 11 Mar 2014 01:30:17 -0400 From: Dave Jones To: Andrew Morton Cc: Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311053017.GB14329@redhat.com> Mail-Followup-To: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Cyrill Gorcunov , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote: > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > > while trying to reproduce a different bug.. > > > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > > > Guys, what stops the migration target page from coming unlocked in > > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > > and page_locked() happen under ptl. > > > > > > I don't see any holes in regular migration. Do you know if this is > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > There probably isn't much point unless trinity is using > sys_move_pages(). Is it? If so it would be interesting to disable > trinity's move_pages calls and see if it still fails. Ok, with move_pages excluded it still oopses. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755078AbaCKM4H (ORCPT ); Tue, 11 Mar 2014 08:56:07 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:29947 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752409AbaCKM4F (ORCPT ); Tue, 11 Mar 2014 08:56:05 -0400 Message-ID: <531F07D4.5000108@oracle.com> Date: Tue, 11 Mar 2014 08:55:48 -0400 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Cyrill Gorcunov , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> In-Reply-To: <20140311053017.GB14329@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: acsinet22.oracle.com [141.146.126.238] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/11/2014 01:30 AM, Dave Jones wrote: > On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote: > > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones wrote: > > > > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote: > > > > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton wrote: > > > > > > > > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain > > > > > > while trying to reproduce a different bug.. > > > > > > > > > > Damn, I thought we'd fixed that but it seems not. Cc's added. > > > > > > > > > > Guys, what stops the migration target page from coming unlocked in > > > > > parallel with zap_pte_range()'s call to migration_entry_to_page()? > > > > > > > > page_table_lock, sort-of. At least, transitions of is_migration_entry() > > > > and page_locked() happen under ptl. > > > > > > > > I don't see any holes in regular migration. Do you know if this is > > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > > > There probably isn't much point unless trinity is using > > sys_move_pages(). Is it? If so it would be interesting to disable > > trinity's move_pages calls and see if it still fails. > > Ok, with move_pages excluded it still oopses. FWIW, yes - I still see both of these issues happening. It's easy to ignore the bad rss-counter, and I've commented out the BUG at swapops.h so that I could keep on testing. There are quite a few issues within mm/ right now, I think there are more than 5 different BUG()s hittable using trinity at this point without a fix. Thanks, Sasha From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754341AbaCKNUa (ORCPT ); Tue, 11 Mar 2014 09:20:30 -0400 Received: from mail-lb0-f182.google.com ([209.85.217.182]:64906 "EHLO mail-lb0-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754154AbaCKNU1 (ORCPT ); Tue, 11 Mar 2014 09:20:27 -0400 Date: Tue, 11 Mar 2014 17:20:24 +0400 From: Cyrill Gorcunov To: Dave Jones Cc: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Sasha Levin , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311132024.GC32390@moon> References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311053017.GB14329@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote: > > > > > > > > I don't see any holes in regular migration. Do you know if this is > > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? > > > > > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. > > > > There probably isn't much point unless trinity is using > > sys_move_pages(). Is it? If so it would be interesting to disable > > trinity's move_pages calls and see if it still fails. > > Ok, with move_pages excluded it still oopses. Dave, is it possible to somehow figure out was someone reading pagemap file at moment of the bug triggering? From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754892AbaCKNXk (ORCPT ); Tue, 11 Mar 2014 09:23:40 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:34880 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753555AbaCKNXj (ORCPT ); Tue, 11 Mar 2014 09:23:39 -0400 Message-ID: <531F0E39.9020100@oracle.com> Date: Tue, 11 Mar 2014 09:23:05 -0400 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Cyrill Gorcunov , Dave Jones CC: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 References: <20140305174503.GA16335@redhat.com> <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> In-Reply-To: <20140311132024.GC32390@moon> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: ucsinet21.oracle.com [156.151.31.93] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/11/2014 09:20 AM, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote: >> > > > >> > > > I don't see any holes in regular migration. Do you know if this is >> > > > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n? >> > > >> > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run. >> > >> > There probably isn't much point unless trinity is using >> > sys_move_pages(). Is it? If so it would be interesting to disable >> > trinity's move_pages calls and see if it still fails. >> >> Ok, with move_pages excluded it still oopses. > > Dave, is it possible to somehow figure out was someone reading pagemap file > at moment of the bug triggering? We can sprinkle printk()s wherever might be useful, might not be 100% accurate but should be close enough to confirm/deny the theory. Thanks, Sasha From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754351AbaCKNmE (ORCPT ); Tue, 11 Mar 2014 09:42:04 -0400 Received: from mail-la0-f46.google.com ([209.85.215.46]:42479 "EHLO mail-la0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752056AbaCKNmB (ORCPT ); Tue, 11 Mar 2014 09:42:01 -0400 Date: Tue, 11 Mar 2014 17:41:58 +0400 From: Cyrill Gorcunov To: Sasha Levin Cc: Dave Jones , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311134158.GD32390@moon> References: <20140305175725.GB16335@redhat.com> <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <531F0E39.9020100@oracle.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: > >> > >>Ok, with move_pages excluded it still oopses. > > > >Dave, is it possible to somehow figure out was someone reading pagemap file > >at moment of the bug triggering? > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but > should be close enough to confirm/deny the theory. After reading some more, I suppose the idea I had is wrong, investigating. Will ping if I find something. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753368AbaCKO2e (ORCPT ); Tue, 11 Mar 2014 10:28:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:6216 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752698AbaCKO2c (ORCPT ); Tue, 11 Mar 2014 10:28:32 -0400 Date: Tue, 11 Mar 2014 10:28:17 -0400 From: Dave Jones To: Cyrill Gorcunov Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311142817.GA26517@redhat.com> Mail-Followup-To: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <20140307002210.GA26603@redhat.com> <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311134158.GD32390@moon> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: > > >> > > >>Ok, with move_pages excluded it still oopses. > > > > > >Dave, is it possible to somehow figure out was someone reading pagemap file > > >at moment of the bug triggering? > > > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but > > should be close enough to confirm/deny the theory. > > After reading some more, I suppose the idea I had is wrong, investigating. > Will ping if I find something. I can rule it out anyway, I can reproduce this by telling trinity to do nothing other than mmap()'s. I'll try and narrow down the exact parameters. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752470AbaCKOh7 (ORCPT ); Tue, 11 Mar 2014 10:37:59 -0400 Received: from mail-lb0-f175.google.com ([209.85.217.175]:43990 "EHLO mail-lb0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751296AbaCKOh5 (ORCPT ); Tue, 11 Mar 2014 10:37:57 -0400 Date: Tue, 11 Mar 2014 18:37:50 +0400 From: Cyrill Gorcunov To: Dave Jones Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311143750.GE32390@moon> References: <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311142817.GA26517@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote: > On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote: > > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: > > > >> > > > >>Ok, with move_pages excluded it still oopses. > > > > > > > >Dave, is it possible to somehow figure out was someone reading pagemap file > > > >at moment of the bug triggering? > > > > > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but > > > should be close enough to confirm/deny the theory. > > > > After reading some more, I suppose the idea I had is wrong, investigating. > > Will ping if I find something. > > I can rule it out anyway, I can reproduce this by telling trinity to do nothing > other than mmap()'s. I'll try and narrow down the exact parameters. Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753458AbaCKO66 (ORCPT ); Tue, 11 Mar 2014 10:58:58 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:26067 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752791AbaCKO65 (ORCPT ); Tue, 11 Mar 2014 10:58:57 -0400 Message-ID: <531F24A6.2020409@oracle.com> Date: Tue, 11 Mar 2014 10:58:46 -0400 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Cyrill Gorcunov , Dave Jones CC: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311024906.GA9191@redhat.com> <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> In-Reply-To: <20140311143750.GE32390@moon> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: acsinet21.oracle.com [141.146.126.237] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/11/2014 10:37 AM, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote: >> On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote: >> > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote: >> > > >> >> > > >>Ok, with move_pages excluded it still oopses. >> > > > >> > > >Dave, is it possible to somehow figure out was someone reading pagemap file >> > > >at moment of the bug triggering? >> > > >> > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but >> > > should be close enough to confirm/deny the theory. >> > >> > After reading some more, I suppose the idea I had is wrong, investigating. >> > Will ping if I find something. >> >> I can rule it out anyway, I can reproduce this by telling trinity to do nothing >> other than mmap()'s. I'll try and narrow down the exact parameters. > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > was passed, right? Share it too please. I've sent one of those last time I reported this issue: https://lkml.org/lkml/2014/1/22/625 Thanks, Sasha From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754910AbaCKRLI (ORCPT ); Tue, 11 Mar 2014 13:11:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:22885 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754492AbaCKRLE (ORCPT ); Tue, 11 Mar 2014 13:11:04 -0400 Date: Tue, 11 Mar 2014 13:10:45 -0400 From: Dave Jones To: Cyrill Gorcunov Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311171045.GA4693@redhat.com> Mail-Followup-To: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <20140310201340.81994295.akpm@linux-foundation.org> <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311143750.GE32390@moon> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 06:37:50PM +0400, Cyrill Gorcunov wrote: > > > After reading some more, I suppose the idea I had is wrong, investigating. > > > Will ping if I find something. > > > > I can rule it out anyway, I can reproduce this by telling trinity to do nothing > > other than mmap()'s. I'll try and narrow down the exact parameters. > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > was passed, right? Share it too please. Hm, I may have been mistaken, and the damage was done by a previous run. I went from being able to reproduce it almost instantly to now not being able to reproduce it at all. Will keep trying. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754626AbaCKRgI (ORCPT ); Tue, 11 Mar 2014 13:36:08 -0400 Received: from mail-la0-f42.google.com ([209.85.215.42]:34096 "EHLO mail-la0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753139AbaCKRgH (ORCPT ); Tue, 11 Mar 2014 13:36:07 -0400 Date: Tue, 11 Mar 2014 21:36:03 +0400 From: Cyrill Gorcunov To: Dave Jones Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311173603.GG32390@moon> References: <20140310214612.3b4de36a.akpm@linux-foundation.org> <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311171045.GA4693@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > > was passed, right? Share it too please. > > Hm, I may have been mistaken, and the damage was done by a previous run. > I went from being able to reproduce it almost instantly to now not being able > to reproduce it at all. Will keep trying. Sasha already gave a link to the syscalls sequence, so no rush. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754977AbaCKRjd (ORCPT ); Tue, 11 Mar 2014 13:39:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:61581 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753019AbaCKRjc (ORCPT ); Tue, 11 Mar 2014 13:39:32 -0400 Date: Tue, 11 Mar 2014 13:39:17 -0400 From: Dave Jones To: Cyrill Gorcunov Cc: Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140311173917.GB4693@redhat.com> Mail-Followup-To: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311173603.GG32390@moon> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > > > was passed, right? Share it too please. > > > > Hm, I may have been mistaken, and the damage was done by a previous run. > > I went from being able to reproduce it almost instantly to now not being able > > to reproduce it at all. Will keep trying. > > Sasha already gave a link to the syscalls sequence, so no rush. It'd be nice to get a more concise reproducer, his list had a little of everything in there. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754643AbaCNM1S (ORCPT ); Fri, 14 Mar 2014 08:27:18 -0400 Received: from mail-lb0-f171.google.com ([209.85.217.171]:35065 "EHLO mail-lb0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754024AbaCNM1Q (ORCPT ); Fri, 14 Mar 2014 08:27:16 -0400 Date: Fri, 14 Mar 2014 16:27:14 +0400 From: Cyrill Gorcunov To: Dave Jones , Sasha Levin Cc: Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140314122714.GR13448@moon> References: <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140311173917.GB4693@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 11, 2014 at 01:39:17PM -0400, Dave Jones wrote: > > > > Sasha already gave a link to the syscalls sequence, so no rush. > > It'd be nice to get a more concise reproducer, his list had a little of everything in there. Dave, could you please send me your config privately so I would try to reproduce the issue locally maybe it shed some light on the problem. Cyrill From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932953AbaCSAkH (ORCPT ); Tue, 18 Mar 2014 20:40:07 -0400 Received: from mail-pa0-f47.google.com ([209.85.220.47]:52147 "EHLO mail-pa0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758599AbaCSAkE (ORCPT ); Tue, 18 Mar 2014 20:40:04 -0400 Date: Tue, 18 Mar 2014 17:38:38 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Dave Jones cc: Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <20140311173917.GB4693@redhat.com> Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 11 Mar 2014, Dave Jones wrote: > On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: > > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > > > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence > > > > was passed, right? Share it too please. > > > > > > Hm, I may have been mistaken, and the damage was done by a previous run. > > > I went from being able to reproduce it almost instantly to now not being able > > > to reproduce it at all. Will keep trying. > > > > Sasha already gave a link to the syscalls sequence, so no rush. > > It'd be nice to get a more concise reproducer, his list had a little of everything in there. I've so far failed to find any explanation for your swapops.h BUG; but believe I have identified one cause for "Bad rss-counter"s. My hunch is that the swapops.h BUG is "nearby", but I just cannot fit it together (the swapops.h BUG comes when rmap cannot find all all the migration entries it inserted earlier: it's a very useful BUG for validating rmap). Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list. Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc. (Cyrill, entirely unrelated, but in preparing this patch I noticed your soft_dirty work in install_file_pte(): which looked good at first, until I realized that it's propagating the soft_dirty of a pte it's about to zap completely, to the unrelated entry it's about to insert in its place. Which seems very odd to me.) [PATCH] mm: fix bad rss-counter if remap_file_pages raced migration Fix some "Bad rss-counter state" reports on exit, arising from the interaction between page migration and remap_file_pages(): zap_pte() must count a migration entry when zapping it. And yes, it is possible (though very unusual) to find an anon page or swap entry in a VM_SHARED nonlinear mapping: coming from that horrid get_user_pages(write, force) case which COWs even in a shared mapping. Signed-off-by: Hugh Dickins --- mm/fremap.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) --- 3.14-rc7/mm/fremap.c 2014-01-19 18:40:07.000000000 -0800 +++ linux/mm/fremap.c 2014-03-18 16:32:39.288612346 -0700 @@ -23,28 +23,44 @@ #include "internal.h" +static int mm_counter(struct page *page) +{ + return PageAnon(page) ? MM_ANONPAGES : MM_FILEPAGES; +} + static void zap_pte(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { pte_t pte = *ptep; + struct page *page; + swp_entry_t entry; if (pte_present(pte)) { - struct page *page; - flush_cache_page(vma, addr, pte_pfn(pte)); pte = ptep_clear_flush(vma, addr, ptep); page = vm_normal_page(vma, addr, pte); if (page) { if (pte_dirty(pte)) set_page_dirty(page); + update_hiwater_rss(mm); + dec_mm_counter(mm, mm_counter(page)); page_remove_rmap(page); page_cache_release(page); + } + } else { /* zap_pte() is not called when pte_none() */ + if (!pte_file(pte)) { update_hiwater_rss(mm); - dec_mm_counter(mm, MM_FILEPAGES); + entry = pte_to_swp_entry(pte); + if (non_swap_entry(entry)) { + if (is_migration_entry(entry)) { + page = migration_entry_to_page(entry); + dec_mm_counter(mm, mm_counter(page)); + } + } else { + free_swap_and_cache(entry); + dec_mm_counter(mm, MM_SWAPENTS); + } } - } else { - if (!pte_file(pte)) - free_swap_and_cache(pte_to_swp_entry(pte)); pte_clear_not_present_full(mm, addr, ptep, 0); } } From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758840AbaCSBKn (ORCPT ); Tue, 18 Mar 2014 21:10:43 -0400 Received: from mail-ve0-f174.google.com ([209.85.128.174]:51476 "EHLO mail-ve0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758530AbaCSBKl (ORCPT ); Tue, 18 Mar 2014 21:10:41 -0400 MIME-Version: 1.0 In-Reply-To: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> Date: Tue, 18 Mar 2014 18:10:40 -0700 X-Google-Sender-Auth: 81iQZDOAwP_n-mD3nFJttN6fI8s Message-ID: Subject: Re: bad rss-counter message in 3.14rc5 From: Linus Torvalds To: Hugh Dickins Cc: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins wrote: > > And yes, it is possible (though very unusual) to find an anon page or > swap entry in a VM_SHARED nonlinear mapping: coming from that horrid > get_user_pages(write, force) case which COWs even in a shared mapping. Hmm. Maybe we could just disallow that forced case. It *used* to be a trivial "we can just do a COW", but that was back when the VM was much simpler and we had no rmap's etc. So "that horrid case" used to be a simple hack that wasn't painful. But I suspect we could very easily just fail it instead of forcing a COW, if that would make it simpler for the VM code. Linus From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758868AbaCSBcx (ORCPT ); Tue, 18 Mar 2014 21:32:53 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:30573 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757980AbaCSBcw (ORCPT ); Tue, 18 Mar 2014 21:32:52 -0400 Message-ID: <5328F3B4.1080208@oracle.com> Date: Tue, 18 Mar 2014 21:32:36 -0400 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Hugh Dickins , Dave Jones CC: Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: acsinet21.oracle.com [141.146.126.237] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/18/2014 08:38 PM, Hugh Dickins wrote: > On Tue, 11 Mar 2014, Dave Jones wrote: >> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: >> > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: >> > > > >> > > > Dave, iirc trinity can write log file pointing which exactly syscall sequence >> > > > was passed, right? Share it too please. >> > > >> > > Hm, I may have been mistaken, and the damage was done by a previous run. >> > > I went from being able to reproduce it almost instantly to now not being able >> > > to reproduce it at all. Will keep trying. >> > >> > Sasha already gave a link to the syscalls sequence, so no rush. >> >> It'd be nice to get a more concise reproducer, his list had a little of everything in there. > > I've so far failed to find any explanation for your swapops.h BUG; > but believe I have identified one cause for "Bad rss-counter"s. > > My hunch is that the swapops.h BUG is "nearby", but I just cannot > fit it together (the swapops.h BUG comes when rmap cannot find all > all the migration entries it inserted earlier: it's a very useful > BUG for validating rmap). > > Untested patch below: I can't quite say Reported-by, because it may > not even be one that you and Sasha have been seeing; but I'm hopeful, > remap_file_pages is in the list. > > Please give this a try, preferably on 3.14-rc or earlier: I've never > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > a lot more than most of us); but have seen them on mmotm/next, so > some other trigger is coming up there, I'll worry about that once > it reaches 3.15-rc. The patch fixed the "Bad rss-counter" errors I've been seeing both in 3.14-rc7 and -next. Thanks, Sasha From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758331AbaCSCGX (ORCPT ); Tue, 18 Mar 2014 22:06:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:26294 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752497AbaCSCGV (ORCPT ); Tue, 18 Mar 2014 22:06:21 -0400 Date: Tue, 18 Mar 2014 22:06:02 -0400 From: Dave Jones To: Sasha Levin Cc: Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319020602.GA29787@redhat.com> Mail-Followup-To: Dave Jones , Sasha Levin , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5328F3B4.1080208@oracle.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote: > > Untested patch below: I can't quite say Reported-by, because it may > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > remap_file_pages is in the list. > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > a lot more than most of us); but have seen them on mmotm/next, so > > some other trigger is coming up there, I'll worry about that once > > it reaches 3.15-rc. > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > 3.14-rc7 and -next. It's looking good here too so far. I'll leave it running overnight to be sure. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758552AbaCSCHF (ORCPT ); Tue, 18 Mar 2014 22:07:05 -0400 Received: from mail-pb0-f45.google.com ([209.85.160.45]:60858 "EHLO mail-pb0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757879AbaCSCHC (ORCPT ); Tue, 18 Mar 2014 22:07:02 -0400 Date: Tue, 18 Mar 2014 19:06:02 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Linus Torvalds cc: Hugh Dickins , Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 18 Mar 2014, Linus Torvalds wrote: > On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins wrote: > > > > And yes, it is possible (though very unusual) to find an anon page or > > swap entry in a VM_SHARED nonlinear mapping: coming from that horrid > > get_user_pages(write, force) case which COWs even in a shared mapping. > > Hmm. Maybe we could just disallow that forced case. > > It *used* to be a trivial "we can just do a COW", but that was back > when the VM was much simpler and we had no rmap's etc. So "that horrid > case" used to be a simple hack that wasn't painful. But I suspect we > could very easily just fail it instead of forcing a COW, if that would > make it simpler for the VM code. I'd love that, if we can get away with it now: depends very much on whether we then turn out to break userspace or not. If I remember correctly, it's been that way since early days, in case ptrace were used to put a breakpoint into a MAP_SHARED mapping of an executable: to prevent that modification from reaching the file, if the file happened to be opened O_RDWR. Usually it's not open for writing, and mapped MAP_PRIVATE anyway. That is still something worth protecting against, I presume; but I'd much rather do it by failing the awkward case, than by perverting the VM to break its own rules. If I'm not mistaken, Konstantin (who happens to be already on this Cc list) had a patch (that I hated) to complicate things, to fix up some of the inconsistencies arising from this very odd and overlooked corner-case. I think he'd prefer this simplification to his patch too. I'll look into it further, but not in haste. Hugh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758413AbaCSCLt (ORCPT ); Tue, 18 Mar 2014 22:11:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45657 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757855AbaCSCLs (ORCPT ); Tue, 18 Mar 2014 22:11:48 -0400 Date: Tue, 18 Mar 2014 22:11:31 -0400 From: Dave Jones To: Sasha Levin , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319021131.GA30018@redhat.com> Mail-Followup-To: Dave Jones , Sasha Levin , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140319020602.GA29787@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote: > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote: > > > > Untested patch below: I can't quite say Reported-by, because it may > > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > > remap_file_pages is in the list. > > > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > > a lot more than most of us); but have seen them on mmotm/next, so > > > some other trigger is coming up there, I'll worry about that once > > > it reaches 3.15-rc. > > > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > > 3.14-rc7 and -next. > > It's looking good here too so far. I'll leave it running overnight to be sure. Of course, that isn't going to happen. Immediately after posting this, I hit the swapops bug. Patch does seem to have cured the bad rss counters though. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758908AbaCSCNK (ORCPT ); Tue, 18 Mar 2014 22:13:10 -0400 Received: from mail-pd0-f171.google.com ([209.85.192.171]:37958 "EHLO mail-pd0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758511AbaCSCNJ (ORCPT ); Tue, 18 Mar 2014 22:13:09 -0400 Date: Tue, 18 Mar 2014 19:12:14 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Sasha Levin cc: Hugh Dickins , Dave Jones , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <5328F3B4.1080208@oracle.com> Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 18 Mar 2014, Sasha Levin wrote: > On 03/18/2014 08:38 PM, Hugh Dickins wrote: > > On Tue, 11 Mar 2014, Dave Jones wrote: > > > On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: > > > > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: > > > > > > > > > > > > Dave, iirc trinity can write log file pointing which exactly > > > syscall sequence > > > > > > was passed, right? Share it too please. > > > > > > > > > > Hm, I may have been mistaken, and the damage was done by a previous > > > run. > > > > > I went from being able to reproduce it almost instantly to now not > > > being able > > > > > to reproduce it at all. Will keep trying. > > > > > > > > Sasha already gave a link to the syscalls sequence, so no rush. > > > > > > It'd be nice to get a more concise reproducer, his list had a little of > > > everything in there. > > > > I've so far failed to find any explanation for your swapops.h BUG; > > but believe I have identified one cause for "Bad rss-counter"s. > > > > My hunch is that the swapops.h BUG is "nearby", but I just cannot > > fit it together (the swapops.h BUG comes when rmap cannot find all > > all the migration entries it inserted earlier: it's a very useful > > BUG for validating rmap). > > > > Untested patch below: I can't quite say Reported-by, because it may > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > remap_file_pages is in the list. > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > a lot more than most of us); but have seen them on mmotm/next, so > > some other trigger is coming up there, I'll worry about that once > > it reaches 3.15-rc. > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > 3.14-rc7 and -next. Great, thanks a lot, Sasha. I was afraid that you'd hit those swapops BUGs, which seemed perhaps to be paired with these; but glad to hear a positive. Let's see how Dave fares. (I've not forgotten shmem fallocate, by the way, but those probably aren't as high on my agenda as you'd like.) Hugh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758388AbaCSCUO (ORCPT ); Tue, 18 Mar 2014 22:20:14 -0400 Received: from mail-pa0-f52.google.com ([209.85.220.52]:36792 "EHLO mail-pa0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751543AbaCSCUM (ORCPT ); Tue, 18 Mar 2014 22:20:12 -0400 Date: Tue, 18 Mar 2014 19:19:09 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Dave Jones cc: Sasha Levin , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <20140319021131.GA30018@redhat.com> Message-ID: References: <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 18 Mar 2014, Dave Jones wrote: > On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote: > > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote: > > > > > > Untested patch below: I can't quite say Reported-by, because it may > > > > not even be one that you and Sasha have been seeing; but I'm hopeful, > > > > remap_file_pages is in the list. > > > > > > > > Please give this a try, preferably on 3.14-rc or earlier: I've never > > > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages > > > > a lot more than most of us); but have seen them on mmotm/next, so > > > > some other trigger is coming up there, I'll worry about that once > > > > it reaches 3.15-rc. > > > > > > The patch fixed the "Bad rss-counter" errors I've been seeing both in > > > 3.14-rc7 and -next. > > > > It's looking good here too so far. I'll leave it running overnight to be sure. > > Of course, that isn't going to happen. Immediately after posting this, I hit the > swapops bug. Patch does seem to have cured the bad rss counters though. Another positive on the rss counters, great, thanks Dave. That encourages me to think again on the swapops BUG, but no promises. Hugh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759005AbaCSCYP (ORCPT ); Tue, 18 Mar 2014 22:24:15 -0400 Received: from mail-ve0-f172.google.com ([209.85.128.172]:45626 "EHLO mail-ve0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758156AbaCSCYN (ORCPT ); Tue, 18 Mar 2014 22:24:13 -0400 MIME-Version: 1.0 In-Reply-To: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> Date: Tue, 18 Mar 2014 19:24:13 -0700 X-Google-Sender-Auth: -fy6gPvZfkbmUNl7X1fFlnUl6K8 Message-ID: Subject: Re: bad rss-counter message in 3.14rc5 From: Linus Torvalds To: Hugh Dickins Cc: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins wrote: > > I'd love that, if we can get away with it now: depends very > much on whether we then turn out to break userspace or not. Right. I suspect we can, though, but it's one of those "we can try it and see". Remind me early in the 3.15 merge window, and we can just turn the "force" case into an error case and see if anybody hollers. > If I remember correctly, it's been that way since early days, > in case ptrace were used to put a breakpoint into a MAP_SHARED > mapping of an executable: to prevent that modification from > reaching the file, if the file happened to be opened O_RDWR. > Usually it's not open for writing, and mapped MAP_PRIVATE anyway. Yes, it's been that way since the very beginning, I think it goes back pretty much as far as MAP_SHARED does. We used to play lots of games wrt MAP_SHARED - in fact I think we used to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for the longest time there was no "true" writable MAP_SHARED at all, but we did have a coherent MAP_PRIVATE and something like the indexer for nntpd wanted a read-only shared mapping of the nntp spool or something like that. I forget the details, it's a _loong_ time ago. So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used to make a lot more sense in that kind of situation, when MAP_SHARED vs MAP_PRIVATE was much less of a black-and-white thing. I really suspect nobody cares wrt ptrace, especially since presumably other systems haven't had those kinds of games (although who knows - HP-UX in particular had some of the shittiest mmap() implementations on the planet - it made even the original Linux mmap hacks look like a thing of pure beauty in comparison). Linus From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758583AbaCSCiD (ORCPT ); Tue, 18 Mar 2014 22:38:03 -0400 Received: from mail-pb0-f44.google.com ([209.85.160.44]:44804 "EHLO mail-pb0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752497AbaCSCiB (ORCPT ); Tue, 18 Mar 2014 22:38:01 -0400 Date: Tue, 18 Mar 2014 19:37:01 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Linus Torvalds cc: Hugh Dickins , Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: Message-ID: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 18 Mar 2014, Linus Torvalds wrote: > On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins wrote: > > > > I'd love that, if we can get away with it now: depends very > > much on whether we then turn out to break userspace or not. > > Right. I suspect we can, though, but it's one of those "we can try it > and see". Remind me early in the 3.15 merge window, and we can just > turn the "force" case into an error case and see if anybody hollers. Super, I'll do that, thanks. For 3.15, and probably 3.16 too, we should keep in place whatever partial accommodations we have for the case (such as allowing for anon and swap in fremap's zap_pte), in case we do need to revert; but clean those away later on. (Not many, I think: it was mainly a guilty secret that VM accounting didn't really hold together.) > > > If I remember correctly, it's been that way since early days, > > in case ptrace were used to put a breakpoint into a MAP_SHARED > > mapping of an executable: to prevent that modification from > > reaching the file, if the file happened to be opened O_RDWR. > > Usually it's not open for writing, and mapped MAP_PRIVATE anyway. > > Yes, it's been that way since the very beginning, I think it goes back > pretty much as far as MAP_SHARED does. > > We used to play lots of games wrt MAP_SHARED - in fact I think we used > to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for > the longest time there was no "true" writable MAP_SHARED at all, but > we did have a coherent MAP_PRIVATE and something like the indexer for > nntpd wanted a read-only shared mapping of the nntp spool or something > like that. I forget the details, it's a _loong_ time ago. > > So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used > to make a lot more sense in that kind of situation, when MAP_SHARED vs > MAP_PRIVATE was much less of a black-and-white thing. > > I really suspect nobody cares wrt ptrace, especially since presumably > other systems haven't had those kinds of games (although who knows - > HP-UX in particular had some of the shittiest mmap() implementations > on the planet - it made even the original Linux mmap hacks look like a > thing of pure beauty in comparison). :) That fits with what I heard of HP-UX mmap, but I never had the pleasure of dealing with it. Hugh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758923AbaCSCm0 (ORCPT ); Tue, 18 Mar 2014 22:42:26 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:33970 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758017AbaCSCmZ (ORCPT ); Tue, 18 Mar 2014 22:42:25 -0400 Message-ID: <53290403.5030809@oracle.com> Date: Tue, 18 Mar 2014 22:42:11 -0400 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Hugh Dickins CC: Dave Jones , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: ucsinet21.oracle.com [156.151.31.93] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/18/2014 10:12 PM, Hugh Dickins wrote: > On Tue, 18 Mar 2014, Sasha Levin wrote: >> On 03/18/2014 08:38 PM, Hugh Dickins wrote: >>> On Tue, 11 Mar 2014, Dave Jones wrote: >>>> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote: >>>> > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote: >>>> > > > >>>> > > > Dave, iirc trinity can write log file pointing which exactly >>>> syscall sequence >>>> > > > was passed, right? Share it too please. >>>> > > >>>> > > Hm, I may have been mistaken, and the damage was done by a previous >>>> run. >>>> > > I went from being able to reproduce it almost instantly to now not >>>> being able >>>> > > to reproduce it at all. Will keep trying. >>>> > >>>> > Sasha already gave a link to the syscalls sequence, so no rush. >>>> >>>> It'd be nice to get a more concise reproducer, his list had a little of >>>> everything in there. >>> >>> I've so far failed to find any explanation for your swapops.h BUG; >>> but believe I have identified one cause for "Bad rss-counter"s. >>> >>> My hunch is that the swapops.h BUG is "nearby", but I just cannot >>> fit it together (the swapops.h BUG comes when rmap cannot find all >>> all the migration entries it inserted earlier: it's a very useful >>> BUG for validating rmap). >>> >>> Untested patch below: I can't quite say Reported-by, because it may >>> not even be one that you and Sasha have been seeing; but I'm hopeful, >>> remap_file_pages is in the list. >>> >>> Please give this a try, preferably on 3.14-rc or earlier: I've never >>> seen "Bad rss-counter"s there myself (trinity uses remap_file_pages >>> a lot more than most of us); but have seen them on mmotm/next, so >>> some other trigger is coming up there, I'll worry about that once >>> it reaches 3.15-rc. >> >> The patch fixed the "Bad rss-counter" errors I've been seeing both in >> 3.14-rc7 and -next. > > Great, thanks a lot, Sasha. I was afraid that you'd hit those swapops > BUGs, which seemed perhaps to be paired with these; but glad to hear > a positive. Let's see how Dave fares. (I've not forgotten shmem > fallocate, by the way, but those probably aren't as high on my agenda > as you'd like.) I do hit the swapops issue a lot, I didn't think that your patch was supposed to fix that so I didn't mention it. Thanks for keeping shmem in mind, I've removed shmem from testing for now but I agree, it's not one of the more important issues to be taken care of. Thanks, Sasha From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758738AbaCSC5D (ORCPT ); Tue, 18 Mar 2014 22:57:03 -0400 Received: from mail-ve0-f182.google.com ([209.85.128.182]:33818 "EHLO mail-ve0-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751543AbaCSC5B (ORCPT ); Tue, 18 Mar 2014 22:57:01 -0400 MIME-Version: 1.0 In-Reply-To: References: <20140311045109.GB12551@redhat.com> <20140310220158.7e8b7f2a.akpm@linux-foundation.org> <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> Date: Tue, 18 Mar 2014 19:57:00 -0700 X-Google-Sender-Auth: Pn_gr7WW8EM-J_jhDu24QaoPyjs Message-ID: Subject: Re: bad rss-counter message in 3.14rc5 From: Linus Torvalds To: Hugh Dickins Cc: Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 7:37 PM, Hugh Dickins wrote: > > For 3.15, and probably 3.16 too, we should keep in place whatever > partial accommodations we have for the case (such as allowing for > anon and swap in fremap's zap_pte), in case we do need to revert; > but clean those away later on. (Not many, I think: it was mainly > a guilty secret that VM accounting didn't really hold together.) Absolutely. See if it works to just stop doing that special COW, and then later on, if we have decided "nobody even noticed", we can remove the hacks we have to support the fact that shared mappings sometimes have anon pages in them. > :) That fits with what I heard of HP-UX mmap, > but I never had the pleasure of dealing with it. They had purely virtually indexed caches, making coherency "interesting". Together with a VM based on some really old BSD VM code that everybody else had thrown out, and that didn't allow you to unmap things partially etc. So HPUX mmap really didn't work, not even for non-shared mmap's. I think they fixed the interfaces in HP-UX 11. But not being coherent meant that the shared mappings tended to still have trouble. nntp largely died, but was replaced with the cyrus imapd that played similar games. At least out mmap was always coherent. Even in MAP_PRIVATE, and with regards to both write() system calls and other mmap PROT_WRITE users. Except when we had bugs. Shared mmap really isn't very simple to get right. Linus From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759168AbaCSLEk (ORCPT ); Wed, 19 Mar 2014 07:04:40 -0400 Received: from cantor2.suse.de ([195.135.220.15]:56856 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759105AbaCSLEj (ORCPT ); Wed, 19 Mar 2014 07:04:39 -0400 Date: Wed, 19 Mar 2014 12:04:36 +0100 From: Jan Kara To: Hugh Dickins Cc: Linus Torvalds , Dave Jones , Cyrill Gorcunov , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319110436.GF26358@quack.suse.cz> References: <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue 18-03-14 19:37:01, Hugh Dickins wrote: > On Tue, 18 Mar 2014, Linus Torvalds wrote: > > On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins wrote: > > > > > > I'd love that, if we can get away with it now: depends very > > > much on whether we then turn out to break userspace or not. > > > > Right. I suspect we can, though, but it's one of those "we can try it > > and see". Remind me early in the 3.15 merge window, and we can just > > turn the "force" case into an error case and see if anybody hollers. > > Super, I'll do that, thanks. > > For 3.15, and probably 3.16 too, we should keep in place whatever > partial accommodations we have for the case (such as allowing for > anon and swap in fremap's zap_pte), in case we do need to revert; > but clean those away later on. (Not many, I think: it was mainly > a guilty secret that VM accounting didn't really hold together.) Different drivers actually use the 'force' argument of get_user_pages() a lot on userspace provided buffers (AFAIU because they want to tell the kernel HW is going to write to that memory so they want to prepare for it). It is hard to imagine someone will use this for MAP_SHARED pages (or what that would be supposed to achieve) but sometimes userspace is surprisingly inventive... Just something to be aware of... Honza -- Jan Kara SUSE Labs, CR From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933892AbaCSME2 (ORCPT ); Wed, 19 Mar 2014 08:04:28 -0400 Received: from mail-la0-f41.google.com ([209.85.215.41]:58484 "EHLO mail-la0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755166AbaCSME0 (ORCPT ); Wed, 19 Mar 2014 08:04:26 -0400 Date: Wed, 19 Mar 2014 16:04:24 +0400 From: Cyrill Gorcunov To: Hugh Dickins Cc: Dave Jones , Sasha Levin , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319120424.GD1728@moon> References: <20140311053017.GB14329@redhat.com> <20140311132024.GC32390@moon> <531F0E39.9020100@oracle.com> <20140311134158.GD32390@moon> <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 05:38:38PM -0700, Hugh Dickins wrote: > > (Cyrill, entirely unrelated, but in preparing this patch I noticed > your soft_dirty work in install_file_pte(): which looked good at > first, until I realized that it's propagating the soft_dirty of a > pte it's about to zap completely, to the unrelated entry it's about > to insert in its place. Which seems very odd to me.) > Thanks a lot Hugh for pointing! I'll revisit all file-softdirty cases. (btw, I've grabbed Dave's config to run trinity and somehow help in testing and attempt to figure out what causes it but didn't yet find hardware node to run, hopefully i'll get a spare machine for testing in a couple of days). From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965391AbaCSOwS (ORCPT ); Wed, 19 Mar 2014 10:52:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:29016 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S964970AbaCSOwR (ORCPT ); Wed, 19 Mar 2014 10:52:17 -0400 Date: Wed, 19 Mar 2014 10:52:00 -0400 From: Dave Jones To: Hugh Dickins Cc: Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140319145200.GA4608@redhat.com> Mail-Followup-To: Dave Jones , Hugh Dickins , Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote: > Another positive on the rss counters, great, thanks Dave. > That encourages me to think again on the swapops BUG, but no promises. So while I slept I ran a test kernel with that swapops BUG replaced with a printk. I'm not sure of the validity of this, given the state of the kernel afterwards is somewhat suspect, but I did see in the logs this morning.. [18728.075153] migration_entry_to_page BUG hit [18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted) [18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted) This might be collateral damage from the swapops thing, I guess we won't know until that gets fixed, but I thought I'd mention that we might still have a problem here. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751722AbaCTFBd (ORCPT ); Thu, 20 Mar 2014 01:01:33 -0400 Received: from mail-pd0-f174.google.com ([209.85.192.174]:38745 "EHLO mail-pd0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750789AbaCTFBb (ORCPT ); Thu, 20 Mar 2014 01:01:31 -0400 Date: Wed, 19 Mar 2014 22:00:29 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Dave Jones cc: Hugh Dickins , Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <20140319145200.GA4608@redhat.com> Message-ID: References: <20140311142817.GA26517@redhat.com> <20140311143750.GE32390@moon> <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 19 Mar 2014, Dave Jones wrote: > On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote: > > > Another positive on the rss counters, great, thanks Dave. > > That encourages me to think again on the swapops BUG, but no promises. > > So while I slept I ran a test kernel with that swapops BUG replaced with a printk. > I'm not sure of the validity of this, given the state of the kernel afterwards > is somewhat suspect, but I did see in the logs this morning.. > > [18728.075153] migration_entry_to_page BUG hit > [18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted) > [18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted) > > This might be collateral damage from the swapops thing, I guess we won't know until > that gets fixed, but I thought I'd mention that we might still have a problem here. Yes, those Bad rss-counters could well be collateral damage from the swapops BUG. To which I believe I now have the answer: again untested, but please give this a try... (It's worth saying, by the way, that these bugs are not a consequence of recent changes at all, they've been there for ages; but trinity has just got better at taunting remap_file_pages and the rest of mm...) [PATCH] mm: fix swapops.h:131 bug if remap_file_pages raced migration Add remove_linear_migration_ptes_from_nonlinear(), to fix an interesting little include/linux/swapops.h:131 BUG_ON(!PageLocked) found by trinity: indicating that remove_migration_ptes() failed to find one of the migration entries that was temporarily inserted. The problem comes from remap_file_pages()'s switch from vma_interval_tree (good for inserting the migration entry) to i_mmap_nonlinear list (no good for locating it again); but can only be a problem if the remap_file_pages() range does not cover the whole of the vma (zap_pte() clears the range). remove_migration_ptes() needs a file_nonlinear method to go down the i_mmap_nonlinear list, applying linear location to look for migration entries in those vmas too, just in case there was this race. The file_nonlinear method does need rmap_walk_control.arg to do this; but it never needed vma passed in - vma comes from its own iteration. Signed-off-by: Hugh Dickins --- include/linux/rmap.h | 3 +-- mm/migrate.c | 32 ++++++++++++++++++++++++++++++++ mm/rmap.c | 5 +++-- 3 files changed, 36 insertions(+), 4 deletions(-) --- 3.14-rc7/include/linux/rmap.h 2014-02-02 18:49:07.429302104 -0800 +++ linux/include/linux/rmap.h 2014-03-19 20:12:27.056451541 -0700 @@ -250,8 +250,7 @@ struct rmap_walk_control { int (*rmap_one)(struct page *page, struct vm_area_struct *vma, unsigned long addr, void *arg); int (*done)(struct page *page); - int (*file_nonlinear)(struct page *, struct address_space *, - struct vm_area_struct *vma); + int (*file_nonlinear)(struct page *, struct address_space *, void *arg); struct anon_vma *(*anon_lock)(struct page *page); bool (*invalid_vma)(struct vm_area_struct *vma, void *arg); }; --- 3.14-rc7/mm/migrate.c 2014-03-16 19:24:19.635512576 -0700 +++ linux/mm/migrate.c 2014-03-19 21:06:02.704527965 -0700 @@ -178,6 +178,37 @@ out: } /* + * Congratulations to trinity for discovering this bug. + * mm/fremap.c's remap_file_pages() accepts any range within a single vma to + * convert that vma to VM_NONLINEAR; and generic_file_remap_pages() will then + * replace the specified range by file ptes throughout (maybe populated after). + * If page migration finds a page within that range, while it's still located + * by vma_interval_tree rather than lost to i_mmap_nonlinear list, no problem: + * zap_pte() clears the temporary migration entry before mmap_sem is dropped. + * But if the migrating page is in a part of the vma outside the range to be + * remapped, then it will not be cleared, and remove_migration_ptes() needs to + * deal with it. Fortunately, this part of the vma is of course still linear, + * so we just need to use linear location on the nonlinear list. + */ +static int remove_linear_migration_ptes_from_nonlinear(struct page *page, + struct address_space *mapping, void *arg) +{ + struct vm_area_struct *vma; + /* hugetlbfs does not support remap_pages, so no huge pgoff worries */ + pgoff_t pgoff = page->index << (PAGE_CACHE_SHIFT - PAGE_SHIFT); + unsigned long addr; + + list_for_each_entry(vma, + &mapping->i_mmap_nonlinear, shared.nonlinear) { + + addr = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT); + if (addr >= vma->vm_start && addr < vma->vm_end) + remove_migration_pte(page, vma, addr, arg); + } + return SWAP_AGAIN; +} + +/* * Get rid of all migration entries and replace them by * references to the indicated page. */ @@ -186,6 +217,7 @@ static void remove_migration_ptes(struct struct rmap_walk_control rwc = { .rmap_one = remove_migration_pte, .arg = old, + .file_nonlinear = remove_linear_migration_ptes_from_nonlinear, }; rmap_walk(new, &rwc); --- 3.14-rc7/mm/rmap.c 2014-02-02 18:49:07.929302115 -0800 +++ linux/mm/rmap.c 2014-03-19 20:16:03.552456686 -0700 @@ -1360,8 +1360,9 @@ static int try_to_unmap_cluster(unsigned } static int try_to_unmap_nonlinear(struct page *page, - struct address_space *mapping, struct vm_area_struct *vma) + struct address_space *mapping, void *arg) { + struct vm_area_struct *vma; int ret = SWAP_AGAIN; unsigned long cursor; unsigned long max_nl_cursor = 0; @@ -1663,7 +1664,7 @@ static int rmap_walk_file(struct page *p if (list_empty(&mapping->i_mmap_nonlinear)) goto done; - ret = rwc->file_nonlinear(page, mapping, vma); + ret = rwc->file_nonlinear(page, mapping, rwc->arg); done: mutex_unlock(&mapping->i_mmap_mutex); From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758999AbaCTNv5 (ORCPT ); Thu, 20 Mar 2014 09:51:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:14980 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757555AbaCTNv4 (ORCPT ); Thu, 20 Mar 2014 09:51:56 -0400 Date: Thu, 20 Mar 2014 09:51:37 -0400 From: Dave Jones To: Hugh Dickins Cc: Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 Message-ID: <20140320135137.GA2263@redhat.com> Mail-Followup-To: Dave Jones , Hugh Dickins , Sasha Levin , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov References: <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote: > > This might be collateral damage from the swapops thing, I guess we won't know until > > that gets fixed, but I thought I'd mention that we might still have a problem here. > > Yes, those Bad rss-counters could well be collateral damage from the > swapops BUG. To which I believe I now have the answer: again untested, > but please give this a try... This survived an overnight run. No swapops bug, and no bad RSS. Good job :) > (It's worth saying, by the way, that these bugs are not a consequence > of recent changes at all, they've been there for ages; but trinity has > just got better at taunting remap_file_pages and the rest of mm...) Indeed. I hope to lift the covers on more stuff like this (and hopefully get it done in a more reproducable manner). A lot of the stuff trinity is doing with VM syscalls is still very naive. Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758670AbaCTOUN (ORCPT ); Thu, 20 Mar 2014 10:20:13 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:36923 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750960AbaCTOUL (ORCPT ); Thu, 20 Mar 2014 10:20:11 -0400 Message-ID: <532AF8E8.8030101@oracle.com> Date: Thu, 20 Mar 2014 10:19:20 -0400 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Dave Jones , Hugh Dickins , Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 References: <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> <20140320135137.GA2263@redhat.com> In-Reply-To: <20140320135137.GA2263@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: ucsinet22.oracle.com [156.151.31.94] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/20/2014 09:51 AM, Dave Jones wrote: > On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote: > > > > This might be collateral damage from the swapops thing, I guess we won't know until > > > that gets fixed, but I thought I'd mention that we might still have a problem here. > > > > Yes, those Bad rss-counters could well be collateral damage from the > > swapops BUG. To which I believe I now have the answer: again untested, > > but please give this a try... > > This survived an overnight run. No swapops bug, and no bad RSS. Good job:) Same here, swapops bug is gone! Thanks, Sasha From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753697AbaCUErz (ORCPT ); Fri, 21 Mar 2014 00:47:55 -0400 Received: from mail-pb0-f41.google.com ([209.85.160.41]:65297 "EHLO mail-pb0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750800AbaCUEry (ORCPT ); Fri, 21 Mar 2014 00:47:54 -0400 Date: Thu, 20 Mar 2014 21:46:51 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Sasha Levin , Dave Jones cc: Cyrill Gorcunov , Andrew Morton , Linux Kernel , linux-mm@kvack.org, Linus Torvalds , Joonsoo Kim , Bob Liu , Konstantin Khlebnikov Subject: Re: bad rss-counter message in 3.14rc5 In-Reply-To: <532AF8E8.8030101@oracle.com> Message-ID: References: <20140311171045.GA4693@redhat.com> <20140311173603.GG32390@moon> <20140311173917.GB4693@redhat.com> <5328F3B4.1080208@oracle.com> <20140319020602.GA29787@redhat.com> <20140319021131.GA30018@redhat.com> <20140319145200.GA4608@redhat.com> <20140320135137.GA2263@redhat.com> <532AF8E8.8030101@oracle.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 20 Mar 2014, Sasha Levin wrote: > On 03/20/2014 09:51 AM, Dave Jones wrote: > > On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote: > > > > > > This might be collateral damage from the swapops thing, I guess we > > won't know until > > > > that gets fixed, but I thought I'd mention that we might still have a > > problem here. > > > > > > Yes, those Bad rss-counters could well be collateral damage from the > > > swapops BUG. To which I believe I now have the answer: again untested, > > > but please give this a try... > > > > This survived an overnight run. No swapops bug, and no bad RSS. Good job:) > > Same here, swapops bug is gone! That was welcome news, thanks guys. I notice it has not (yet) magically appeared in Linus's public tree like the rss one did: so to be on the safe side, I'll just repost it now, with your Reported-and-tested-bys, otherwise unchanged. Hugh