From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934709AbdEVO3m (ORCPT ); Mon, 22 May 2017 10:29:42 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:45593 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933917AbdEVO3i (ORCPT ); Mon, 22 May 2017 10:29:38 -0400 Date: Mon, 22 May 2017 17:29:28 +0300 From: Mike Rapoport To: Michal Hocko Cc: "Kirill A. Shutemov" , Andrew Morton , Arnd Bergmann , "Kirill A. Shutemov" , Andrea Arcangeli , Pavel Emelyanov , linux-mm , lkml Subject: Re: [PATCH] mm: introduce MADV_CLR_HUGEPAGE References: <1495433562-26625-1-git-send-email-rppt@linux.vnet.ibm.com> <20170522114243.2wrdbncilozygbpl@node.shutemov.name> <20170522133559.GE27382@rapoport-lnx> <20170522135548.GA8514@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170522135548.GA8514@dhcp22.suse.cz> User-Agent: Mutt/1.5.24 (2015-08-30) X-TM-AS-GCONF: 00 x-cbid: 17052214-0020-0000-0000-000003708B5B X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052214-0021-0000-0000-000041D70AAB Message-Id: <20170522142927.GG27382@rapoport-lnx> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-05-22_09:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705220077 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, May 22, 2017 at 03:55:48PM +0200, Michal Hocko wrote: > On Mon 22-05-17 16:36:00, Mike Rapoport wrote: > > On Mon, May 22, 2017 at 02:42:43PM +0300, Kirill A. Shutemov wrote: > > > On Mon, May 22, 2017 at 09:12:42AM +0300, Mike Rapoport wrote: > > > > Currently applications can explicitly enable or disable THP for a memory > > > > region using MADV_HUGEPAGE or MADV_NOHUGEPAGE. However, once either of > > > > these advises is used, the region will always have > > > > VM_HUGEPAGE/VM_NOHUGEPAGE flag set in vma->vm_flags. > > > > The MADV_CLR_HUGEPAGE resets both these flags and allows managing THP in > > > > the region according to system-wide settings. > > > > > > Seems reasonable. But could you describe an use-case when it's useful in > > > real world. > > > > My use-case was combination of pre- and post-copy migration of containers > > with CRIU. > > In this case we populate a part of a memory region with data that was saved > > during the pre-copy stage. Afterwards, the region is registered with > > userfaultfd and we expect to get page faults for the parts of the region > > that were not yet populated. However, khugepaged collapses the pages and > > the page faults we would expect do not occur. > > I am not sure I undestand the problem. Do I get it right that the > khugepaged will effectivelly corrupt the memory by collapsing a range > which is not yet fully populated? If yes shouldn't that be fixed in > khugepaged rather than adding yet another madvise command? Also how do > you prevent on races? (say you VM_NOHUGEPAGE, khugepaged would be in the > middle of the operation and sees a collapsable vma and you get the same > result) Probably I didn't explained it too well. The range is intentionally not populated. When we combine pre- and post-copy for process migration, we create memory pre-dump without stopping the process, then we freeze the process without dumping the pages it has dirtied between pre-dump and freeze, and then, during restore, we populate the dirtied pages using userfaultfd. When CRIU restores a process in such scenario, it does something like: * mmap() memory region * fill in the pages that were collected during the pre-dump * do some other stuff * register memory region with userfaultfd * populate the missing memory on demand khugepaged collapses the pages in the partially populated regions before we have a chance to register these regions with userfaultfd, which would prevent the collapse. We could have used MADV_NOHUGEPAGE right after the mmap() call, and then there would be no race because there would be nothing for khugepaged to collapse at that point. But the problem is that we have no way to reset *HUGEPAGE flags after the memory restore is complete. > -- > Michal Hocko > SUSE Labs -- Sincerely yours, Mike.