From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mike Rapoport Subject: Re: [PATCH] mm: introduce MADV_CLR_HUGEPAGE Date: Wed, 24 May 2017 13:39:48 +0300 Message-ID: <20170524103947.GC3063@rapoport-lnx> References: <1495433562-26625-1-git-send-email-rppt@linux.vnet.ibm.com> <20170522114243.2wrdbncilozygbpl@node.shutemov.name> <20170522133559.GE27382@rapoport-lnx> <20170522135548.GA8514@dhcp22.suse.cz> <20170522142927.GG27382@rapoport-lnx> <20170524075043.GB3063@rapoport-lnx> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org To: Vlastimil Babka Cc: Michal Hocko , "Kirill A. Shutemov" , Andrew Morton , Arnd Bergmann , "Kirill A. Shutemov" , Andrea Arcangeli , Pavel Emelyanov , linux-mm , lkml , Linux API List-Id: linux-api@vger.kernel.org On Wed, May 24, 2017 at 09:58:06AM +0200, Vlastimil Babka wrote: > On 05/24/2017 09:50 AM, Mike Rapoport wrote: > > On Mon, May 22, 2017 at 05:52:47PM +0200, Vlastimil Babka wrote: > >> On 05/22/2017 04:29 PM, Mike Rapoport wrote: > >>> > >>> Probably I didn't explained it too well. > >>> > >>> The range is intentionally not populated. When we combine pre- and > >>> post-copy for process migration, we create memory pre-dump without stopping > >>> the process, then we freeze the process without dumping the pages it has > >>> dirtied between pre-dump and freeze, and then, during restore, we populate > >>> the dirtied pages using userfaultfd. > >>> > >>> When CRIU restores a process in such scenario, it does something like: > >>> > >>> * mmap() memory region > >>> * fill in the pages that were collected during the pre-dump > >>> * do some other stuff > >>> * register memory region with userfaultfd > >>> * populate the missing memory on demand > >>> > >>> khugepaged collapses the pages in the partially populated regions before we > >>> have a chance to register these regions with userfaultfd, which would > >>> prevent the collapse. > >>> > >>> We could have used MADV_NOHUGEPAGE right after the mmap() call, and then > >>> there would be no race because there would be nothing for khugepaged to > >>> collapse at that point. But the problem is that we have no way to reset > >>> *HUGEPAGE flags after the memory restore is complete. > >> > >> Hmm, I wouldn't be that sure if this is indeed race-free. Check that > >> this scenario is indeed impossible? > >> > >> - you do the mmap > >> - khugepaged will choose the process' mm to scan > >> - khugepaged will get to the vma in question, it doesn't have > >> MADV_NOHUGEPAGE yet > >> - you set MADV_NOHUGEPAGE on the vma > >> - you start populating the vma > >> - khugepaged sees the vma is non-empty, collapses > >> > >> unless I'm wrong, the racers will have mmap_sem for reading only when > >> setting/checking the MADV_NOHUGEPAGE? Might be actually considered a bug. > >> > >> However, can't you use prctl(PR_SET_THP_DISABLE) instead? "If arg2 has a > >> nonzero value, the flag is set, otherwise it is cleared." says the > >> manpage. Do it before the mmap and you avoid the race as well? > > > > Unfortunately, prctl(PR_SET_THP_DISABLE) didn't help :( > > When I've tried to use it, I've ended up with VM_NOHUGEPAGE set on all VMAs > > created after prctl(). This returns me to the state when checkpoint-restore > > alters the application vma->vm_flags although it shouldn't and I do not see > > a way to fix it using existing interfaces. > > [CC linux-api, should have been done in the initial posting already] Sorry, missed that. > Hm so the prctl does: > > if (arg2) > me->mm->def_flags |= VM_NOHUGEPAGE; > else > me->mm->def_flags &= ~VM_NOHUGEPAGE; > > That's rather lazy implementation IMHO. Could we change it so the flag > is stored elsewhere in the mm, and the code that decides to (not) use > THP will check both the per-vma flag and the per-mm flag? I afraid I don't understand how that can help. What we need is an ability to temporarily disable collapse of the pages in VMAs that do not have VM_*HUGEPAGE flags set and that after we re-enable THP, the vma->vm_flags for those VMAs will remain intact. -- Sincerely yours, Mike. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org