From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38381) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XbXOE-0006CK-Li for qemu-devel@nongnu.org; Tue, 07 Oct 2014 12:13:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XbXO8-0002Dd-Vi for qemu-devel@nongnu.org; Tue, 07 Oct 2014 12:13:30 -0400 Received: from mail-vc0-x249.google.com ([2607:f8b0:400c:c03::249]:36844) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XbXO8-0002DF-OD for qemu-devel@nongnu.org; Tue, 07 Oct 2014 12:13:24 -0400 Received: by mail-vc0-f201.google.com with SMTP id hq11so524826vcb.2 for ; Tue, 07 Oct 2014 09:13:23 -0700 (PDT) Date: Tue, 7 Oct 2014 09:13:20 -0700 From: Peter Feiner Message-ID: <20141007161320.GA17858@google.com> References: <1412356087-16115-1-git-send-email-aarcange@redhat.com> <1412356087-16115-11-git-send-email-aarcange@redhat.com> <20141006085540.GD2336@work-vm> <20141006164156.GA31075@redhat.com> <20141007141913.GC2342@redhat.com> <20141007155247.GD2342@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141007155247.GD2342@redhat.com> Subject: Re: [Qemu-devel] [PATCH 10/17] mm: rmap preparation for remap_anon_pages List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Andrea Arcangeli Cc: Robert Love , Dave Hansen , Jan Kara , KVM list , Neil Brown , Stefan Hajnoczi , qemu-devel@nongnu.org, linux-mm , KOSAKI Motohiro , Michel Lespinasse , Taras Glek , Andrew Jones , Juan Quintela , Hugh Dickins , Isaku Yamahata , Mel Gorman , Sasha Levin , Android Kernel Team , "Dr. David Alan Gilbert" , "Huangpeng (Peter)" , Andres Lagar-Cavilla , Christopher Covington , Anthony Liguori , Mike Hommey , Keith Packard , Wenchao Xia , Linux API , Linux Kernel Mailing List , Andy Lutomirski , Minchan Kim , Dmitry Adamushko , Johannes Weiner , Paolo Bonzini , Andrew Morton , Linus Torvalds On Tue, Oct 07, 2014 at 05:52:47PM +0200, Andrea Arcangeli wrote: > I probably grossly overestimated the benefits of resolving the > userfault with a zerocopy page move, sorry. [...] For posterity, I think it's worth noting that most expensive aspect of a TLB shootdown is the interprocessor interrupt necessary to flush other CPUs' TLBs. On a many-core machine, copying 4K of data looks pretty cheap compared to taking an interrupt and invalidating TLBs on many cores :-) > [...] So if we entirely drop the > zerocopy behavior and the TLB flush of the old page like you > suggested, the way to keep the userfaultfd mechanism decoupled from > the userfault resolution mechanism would be to implement an > atomic-copy syscall. That would work for SIGBUS userfaults too without > requiring a pseudofd then. It would be enough then to call > mcopy_atomic(userfault_addr,tmp_addr,len) with the only constraints > that len must be a multiple of PAGE_SIZE. Of course mcopy_atomic > wouldn't page fault or call GUP into the destination address (it can't > otherwise the in-flight partial copy would be visible to the process, > breaking the atomicity of the copy), but it would fill in the > pte/trans_huge_pmd with the same strict behavior that remap_anon_pages > currently has (in turn it would by design bypass the VM_USERFAULT > check and be ideal for resolving userfaults). > > mcopy_atomic could then be also extended to tmpfs and it would work > without requiring the source page to be a tmpfs page too without > having to convert page types on the fly. > > If I add mcopy_atomic, the patch in subject (10/17) can be dropped of > course so it'd be even less intrusive than the current > remap_anon_pages and it would require zero TLB flush during its > runtime (it would just require an atomic copy). I like this new approach. It will be good to have a single interface for resolving anon and tmpfs userfaults. > So should I try to embed a mcopy_atomic inside userfault_write or can > I expose it to userland as a standalone new syscall? Or should I do > something different? Comments? One interesting (ab)use of userfault_write would be that the faulting process and the fault-handling process could be different, which would be necessary for post-copy live migration in CRIU (http://criu.org). Aside from the asthetic difference, I can't think of any advantage in favor of a syscall. Peter