From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f46.google.com (mail-qa0-f46.google.com [209.85.216.46]) by kanga.kvack.org (Postfix) with ESMTP id 070BB6B0036 for ; Tue, 6 May 2014 12:18:46 -0400 (EDT) Received: by mail-qa0-f46.google.com with SMTP id w8so8667940qac.5 for ; Tue, 06 May 2014 09:18:46 -0700 (PDT) Received: from mail-qa0-x235.google.com (mail-qa0-x235.google.com [2607:f8b0:400d:c00::235]) by mx.google.com with ESMTPS id 35si2411233qgy.36.2014.05.06.09.18.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 06 May 2014 09:18:45 -0700 (PDT) Received: by mail-qa0-f53.google.com with SMTP id ih12so4820111qab.40 for ; Tue, 06 May 2014 09:18:45 -0700 (PDT) Date: Tue, 6 May 2014 12:18:38 -0400 From: Jerome Glisse Subject: Re: [RFC] Heterogeneous memory management (mirror process address space on a device mmu). Message-ID: <20140506161836.GC6731@gmail.com> References: <1399038730-25641-1-git-send-email-j.glisse@gmail.com> <20140506102925.GD11096@twins.programming.kicks-ass.net> <20140506150014.GA6731@gmail.com> <20140506153315.GB6731@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Linus Torvalds Cc: Peter Zijlstra , linux-mm , Linux Kernel Mailing List , linux-fsdevel , Mel Gorman , "H. Peter Anvin" , Andrew Morton , Linda Wang , Kevin E Martin , Jerome Glisse , Andrea Arcangeli , Johannes Weiner , Larry Woodman , Rik van Riel , Dave Airlie , Jeff Law , Brendan Conoboy , Joe Donohue , Duncan Poole , Sherry Cheung , Subhash Gutti , John Hubbard , Mark Hairgrove , Lucien Dunning , Cameron Buschardt , Arvind Gopalakrishnan , Haggai Eran , Or Gerlitz , Sagi Grimberg , Shachar Raindel , Liran Liss , Roland Dreier , "Sander, Ben" , "Stoner, Greg" , "Bridgman, John" , "Mantor, Michael" , "Blinzer, Paul" , "Morichetti, Laurent" , "Deucher, Alexander" , "Gabbay, Oded" , Davidlohr Bueso On Tue, May 06, 2014 at 08:47:48AM -0700, Linus Torvalds wrote: > On Tue, May 6, 2014 at 8:33 AM, Jerome Glisse wrote: > > > > So how can i solve the issue at hand. A device that has its own page > > table and can not mirror the cpu page table, nor can the device page > > table be updated atomicly from the cpu. > > So? Just model it as a TLB. > > Sure, the TLB is slow and crappy and is in external memory rather than > on-die, but it's still a TLB. > > We have CPU's that do that kind of crazy thing (powerpc and sparc both > have these kinds of "in-memory TLB extensions" in addition to the > on-die TLB, they just call them "inverse page tables" to try to fool > people about what they are). > > > I understand that we do not want to sleep when updating process cpu > > page table but note that only process that use the gpu would have to > > sleep. So only process that can actually benefit from the using GPU > > will suffer the consequences. > > NO! > > You don't get it. If a callback can sleep, then we cannot protect it > with a spinlock. > > It doesn't matter if it only sleeps once in a millennium. It still > forces its crap on the rest of the system. I do understand that i was pointing out that if i move to, tlb which i am fine with, i will still need to sleep there. That's all i wanted to stress, i did not wanted force using mmu_notifier, i am fine with them becoming atomic as long as i have a place where i can intercept cpu page table update and propagate them to device mmu. > > So there is no way in hell that we will allow that VM notifier crap. None. > > And as I've mentioned, there is a correct place to slot this in, and > that correct way is the _only_ way to ever support future GPU's that > _do_ share direct access to the page tables. This work was done in cooperation with NVidia and we discussed with AMD too so i am very much aware of what is coming next on hardware front and being able to have GPU have their own GPU page table ie not walking the CPU one is something of interest to the people who design those future generation of GPU. > > So trying to do it any other way is broken _anyway_. > > Linus I will respin without using mmu_notifier and by hooking it as tlb shootdown. But it will still need to sleep during the device tlb shootdown and that's the point i want to make sure is clear. Cheers, Jerome -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org