From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [PATCH hmm 02/15] mm/mmu_notifier: add an interval tree notifier Date: Mon, 21 Oct 2019 15:47:48 -0400 Message-ID: <20191021194748.GA5644@redhat.com> References: <20191015181242.8343-1-jgg@ziepe.ca> <20191015181242.8343-3-jgg@ziepe.ca> <20191021183056.GA3177@redhat.com> <20191021185421.GG6285@mellanox.com> <20191021191157.GA5208@redhat.com> <20191021192448.GK6285@mellanox.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20191021192448.GK6285-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> Content-Disposition: inline List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: Jason Gunthorpe Cc: Andrea Arcangeli , Ralph Campbell , "linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , John Hubbard , "Felix.Kuehling-5C7GfCeVMHo@public.gmane.org" , "dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org" , Michal Hocko , "linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org" , "amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org" , Ben Skeggs T24gTW9uLCBPY3QgMjEsIDIwMTkgYXQgMDc6MjQ6NTNQTSArMDAwMCwgSmFzb24gR3VudGhvcnBl IHdyb3RlOgo+IE9uIE1vbiwgT2N0IDIxLCAyMDE5IGF0IDAzOjExOjU3UE0gLTA0MDAsIEplcm9t ZSBHbGlzc2Ugd3JvdGU6Cj4gPiA+IFNpbmNlIHRoYXQgcmVhZGVyIGlzIG5vdCBsb2NrZWQgd2Ug bmVlZCByZWxlYXNlIHNlbWFudGljcyBoZXJlIHRvCj4gPiA+IGVuc3VyZSB0aGUgdW5sb2NrZWQg cmVhZGVyIHNlZXMgYSBmdWxseSBpbml0aW5hbGl6ZWQgbW11X25vdGlmaWVyX21tCj4gPiA+IHN0 cnVjdHVyZSB3aGVuIGl0IG9ic2VydmVzIHRoZSBwb2ludGVyLgo+ID4gCj4gPiBJIHRob3VnaHQg dGhlIG1tX3Rha2VfYWxsX2xvY2tzKCkgd291bGQgaGF2ZSBoYWQgYSBiYXJyaWVyIGFuZCB0aHVz Cj4gPiB0aGF0IHlvdSBjb3VsZCBub3Qgc2VlIG1tdV9ub3RpZmllciBzdHJ1Y3QgcGFydGlhbHkg aW5pdGlhbGl6ZWQuIAo+IAo+IE5vdCBzdXJlLCB1c3VhbGx5IGEgbG9jayBhY3F1aXJlIGRvZXNu J3QgaGF2ZSBhIHN0b3JlIGJhcnJpZXI/CgpZZWFoIGxpa2VseS4KCj4gCj4gRXZlbiBpZiBpdCBk aWQsIHdlIHdvdWxkIHN0aWxsIG5lZWQgc29tZSBwYWlyaW5nIHJlYWQgYmFycmllci4uCj4gCj4g PiBoYXZpbmcgdGhlIGFjcXVpcmUvcmVsZWFzZSBhcyBzYWZldHkgbmV0IGRvZXMgbm90IGh1cnQu IE1heWJlIGFkZCBhCj4gPiBjb21tZW50IGFib3V0IHRoZSBzdHJ1Y3QgaW5pdGlhbGl6YXRpb24g bmVlZGluZyB0byBiZSB2aXNpYmxlIGJlZm9yZQo+ID4gcG9pbnRlciBpcyBzZXQuCj4gCj4gSXMg dGhpcyBjbGVhcj8KPiAKPiAgICAgICAgICAqIHJlbGVhc2Ugc2VtYW50aWNzIG9uIHRoZSBpbml0 aWFsaXphdGlvbiBvZiB0aGUKPiAgICAgICAgICAqIG1tdV9ub3RpZmllcl9tbSdzIGNvbnRlbnRz IGFyZSBwcm92aWRlZCBmb3IgdW5sb2NrZWQgcmVhZGVycy4KPiAJICogYWNxdWlyZSBjYW4gb25s eSBiZSB1c2VkIHdoaWxlIGhvbGRpbmcgdGhlCj4gICAgICAgICAgKiBtbWdyYWIgb3IgbW1nZXQs IGFuZCBpcyBzYWZlIGJlY2F1c2Ugb25jZSBjcmVhdGVkIHRoZQo+ICAgICAgICAgICogbW11X25v dGlmaWZlcl9tbSBpcyBub3QgZnJlZWQgdW50aWwgdGhlIG1tIGlzIGRlc3Ryb3llZC4KPiAgICAg ICAgICAqIFVzZXJzIGhvbGRpbmcgdGhlIG1tYXBfc2VtIG9yIG9uZSBvZiB0aGUKPiAJICogbW1f dGFrZV9hbGxfbG9ja3MoKSBkbyBub3QgbmVlZCB0byB1c2UgYWNxdWlyZSBzZW1hbnRpY3MuCj4g Cj4gSXQgYWxzbyBoZWxwcyBleHBsYWluIHdoeSB0aGVyZSBpcyBubyBsb2NraW5nIGFyb3VuZCB0 aGUgb3RoZXIKPiByZWFkZXJzLCB3aGljaCBoYXMgcHV6emxlZCBtZSBpbiB0aGUgcGFzdCBhdCBs ZWFzdC4KClBlcmZlY3QuCgpKw6lyw7RtZQoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18KYW1kLWdmeCBtYWlsaW5nIGxpc3QKYW1kLWdmeEBsaXN0cy5mcmVl ZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m by9hbWQtZ2Z4 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9026CA9EAF for ; Mon, 21 Oct 2019 19:47:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C2CA92086D for ; Mon, 21 Oct 2019 19:47:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FgXx8MPj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729270AbfJUTr5 (ORCPT ); Mon, 21 Oct 2019 15:47:57 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:23667 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726672AbfJUTr4 (ORCPT ); Mon, 21 Oct 2019 15:47:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1571687275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=61Nnku0I+13ll91wxHp2GCqWsterOpYjidrMjRQ6FOk=; b=FgXx8MPj2E5264FKUKvWQbruJp1cI/6JkRubU1VJQfFvoubEx+GLp/4P4dgRAKSQ/omar4 T6vDwlwnUHe6jTlViX5l/cqIB4V2yaJvOMZf6PbtjnZ8+uV5Ccx+CWIh9f/GjiISwr9mtF E5+Ghp2essq6A+MdDcmipjIu6p0BDv0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-184-tvVLh5QnNemqg5-Lf0njnA-1; Mon, 21 Oct 2019 15:47:52 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D6C07800D41; Mon, 21 Oct 2019 19:47:50 +0000 (UTC) Received: from redhat.com (unknown [10.20.6.178]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DB9105C219; Mon, 21 Oct 2019 19:47:49 +0000 (UTC) Date: Mon, 21 Oct 2019 15:47:48 -0400 From: Jerome Glisse To: Jason Gunthorpe Cc: Ralph Campbell , John Hubbard , "Felix.Kuehling@amd.com" , "linux-rdma@vger.kernel.org" , "linux-mm@kvack.org" , Andrea Arcangeli , "dri-devel@lists.freedesktop.org" , "amd-gfx@lists.freedesktop.org" , Ben Skeggs , Michal Hocko Subject: Re: [PATCH hmm 02/15] mm/mmu_notifier: add an interval tree notifier Message-ID: <20191021194748.GA5644@redhat.com> References: <20191015181242.8343-1-jgg@ziepe.ca> <20191015181242.8343-3-jgg@ziepe.ca> <20191021183056.GA3177@redhat.com> <20191021185421.GG6285@mellanox.com> <20191021191157.GA5208@redhat.com> <20191021192448.GK6285@mellanox.com> MIME-Version: 1.0 In-Reply-To: <20191021192448.GK6285@mellanox.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: tvVLh5QnNemqg5-Lf0njnA-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Mon, Oct 21, 2019 at 07:24:53PM +0000, Jason Gunthorpe wrote: > On Mon, Oct 21, 2019 at 03:11:57PM -0400, Jerome Glisse wrote: > > > Since that reader is not locked we need release semantics here to > > > ensure the unlocked reader sees a fully initinalized mmu_notifier_mm > > > structure when it observes the pointer. > >=20 > > I thought the mm_take_all_locks() would have had a barrier and thus > > that you could not see mmu_notifier struct partialy initialized.=20 >=20 > Not sure, usually a lock acquire doesn't have a store barrier? Yeah likely. >=20 > Even if it did, we would still need some pairing read barrier.. >=20 > > having the acquire/release as safety net does not hurt. Maybe add a > > comment about the struct initialization needing to be visible before > > pointer is set. >=20 > Is this clear? >=20 > * release semantics on the initialization of the > * mmu_notifier_mm's contents are provided for unlocked readers. > =09 * acquire can only be used while holding the > * mmgrab or mmget, and is safe because once created the > * mmu_notififer_mm is not freed until the mm is destroyed. > * Users holding the mmap_sem or one of the > =09 * mm_take_all_locks() do not need to use acquire semantics. >=20 > It also helps explain why there is no locking around the other > readers, which has puzzled me in the past at least. Perfect. J=E9r=F4me