From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D61F6C47083 for ; Wed, 2 Jun 2021 09:11:25 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 475F16101C for ; Wed, 2 Jun 2021 09:11:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 475F16101C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id C6FC240617; Wed, 2 Jun 2021 05:11:24 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aWQjLNX2pVqj; Wed, 2 Jun 2021 05:11:19 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 28B4949F83; Wed, 2 Jun 2021 05:11:19 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 8E03649F8F for ; Wed, 2 Jun 2021 05:11:18 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FJi7xE8p5-Y3 for ; Wed, 2 Jun 2021 05:11:14 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 1831A49F83 for ; Wed, 2 Jun 2021 05:11:14 -0400 (EDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AD0FB608FE; Wed, 2 Jun 2021 09:11:12 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loMu6-004zLH-Ha; Wed, 02 Jun 2021 10:11:10 +0100 Date: Wed, 02 Jun 2021 10:11:09 +0100 Message-ID: <87a6o81viq.wl-maz@kernel.org> From: Marc Zyngier To: Alex Williamson Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA In-Reply-To: <20210504120348.2eec075b@redhat.com> References: <20210429162906.32742-1-sdonthineni@nvidia.com> <20210429162906.32742-2-sdonthineni@nvidia.com> <20210429122840.4f98f78e@redhat.com> <470360a7-0242-9ae5-816f-13608f957bf6@nvidia.com> <20210429134659.321a5c3c@redhat.com> <87czucngdc.wl-maz@kernel.org> <1edb2c4e-23f0-5730-245b-fc6d289951e1@nvidia.com> <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> <20210504120348.2eec075b@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: alex.williamson@redhat.com, vsethi@nvidia.com, mark.kettenis@xs4all.nl, sdonthineni@nvidia.com, will@kernel.org, catalin.marinas@arm.com, christoffer.dall@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, jsequeira@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , "catalin.marinas@arm.com" , Shanker Donthineni , Vikram Sethi , Jason Sequeira , "will@kernel.org" , "kvmarm@lists.cs.columbia.edu" , "linux-arm-kernel@lists.infradead.org" , Mark Kettenis X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu T24gVHVlLCAwNCBNYXkgMjAyMSAxOTowMzo0OCArMDEwMCwKQWxleCBXaWxsaWFtc29uIDxhbGV4 LndpbGxpYW1zb25AcmVkaGF0LmNvbT4gd3JvdGU6Cj4gCj4gT24gTW9uLCAzIE1heSAyMDIxIDIy OjAzOjU5ICswMDAwCj4gVmlrcmFtIFNldGhpIDx2c2V0aGlAbnZpZGlhLmNvbT4gd3JvdGU6Cj4g Cj4gPiBIaSBBbGV4LAo+ID4gPiBGcm9tOiBBbGV4IFdpbGxpYW1zb24gPGFsZXgud2lsbGlhbXNv bkByZWRoYXQuY29tPgo+ID4gPiBPbiBNb24sIDMgTWF5IDIwMjEgMTM6NTk6NDMgKzAwMDAKPiA+ ID4gVmlrcmFtIFNldGhpIDx2c2V0aGlAbnZpZGlhLmNvbT4gd3JvdGU6ICAKPiA+ID4gPiA+IEZy b206IE1hcmsgS2V0dGVuaXMgPG1hcmsua2V0dGVuaXNAeHM0YWxsLm5sPiAgCj4gPiA+ID4gPiA+ IEZyb206IE1hcmMgWnluZ2llciA8bWF6QGtlcm5lbC5vcmc+ICAKPiA+ID4gPgo+ID4gPiA+IHNu aXAgIAo+ID4gPiA+ID4gPiBJZiwgYnkgZW51bWVyYXRpbmcgdGhlIHByb3BlcnRpZXMgb2YgUHJl ZmV0Y2hhYmxlLCB5b3UgY2FuIHNob3cKPiA+ID4gPiA+ID4gdGhhdCB0aGV5IGFyZSBhIHN0cmlj dCBzdXBlcnNldCBvZiBOb3JtYWxfTkMsIEknbSBvbiBib2FyZC4gSQo+ID4gPiA+ID4gPiBoYXZl bid0IHNlZW4gc3VjaCBhbiBlbnVtZXJhdGlvbiBzbyBmYXIuCj4gPiA+ID4gPiA+ICAKPiA+ID4g PiBzbmlwICAKPiA+ID4gPiA+ID4gUmlnaHQsIHNvIHdlIGhhdmUgbWFkZSBhIHNtYWxsIHN0ZXAg aW4gdGhlIGRpcmVjdGlvbiBvZiBtYXBwaW5nCj4gPiA+ID4gPiA+ICJwcmVmZXRjaGFibGUiIG9u dG8gIk5vcm1hbF9OQyIsIHRoYW5rcyBmb3IgdGhhdC4gV2hhdCBhYm91dCBhbGwKPiA+ID4gPiA+ ID4gdGhlIG90aGVyIHByb3BlcnRpZXMgKHVuYWxpZ25lZCBhY2Nlc3Nlcywgb3JkZXJpbmcsIGdh dGhlcmluZyk/ICAKPiA+ID4gPiA+ICAKPiA+ID4gPiBSZWdhcmRpbmcgZ2F0aGVyaW5nL3dyaXRl IGNvbWJpbmluZywgdGhhdCBpcyBhbHNvIGFsbG93ZWQgdG8KPiA+ID4gPiBwcmVmZXRjaGFibGUg cGVyIFBDSSBzcGVjICAKPiA+ID4gCj4gPiA+IEFzIG90aGVycyBoYXZlIHN0YXRlZCwgZ2F0aGVy L3dyaXRlIGNvbWJpbmluZyBpdHNlbGYgaXMgbm90IHdlbGwgZGVmaW5lZC4KPiA+ID4gICAKPiA+ ID4gPiBGcm9tIDEuMy4yLjIgb2YgNS8wIGJhc2Ugc3BlYzoKPiA+ID4gPiBBIFBDSSBFeHByZXNz IEVuZHBvaW50IHJlcXVlc3RpbmcgbWVtb3J5IHJlc291cmNlcyB0aHJvdWdoIGEgQkFSIG11c3QK PiA+ID4gPiBzZXQgdGhlIEJBUidzIFByZWZldGNoYWJsZSBiaXQgdW5sZXNzIHRoZSByYW5nZSBj b250YWlucyBsb2NhdGlvbnMKPiA+ID4gPiB3aXRoIHJlYWQgc2lkZS1lZmZlY3RzIG9yIGxvY2F0 aW9ucyBpbiB3aGljaCB0aGUgRnVuY3Rpb24gZG9lcyBub3QgdG9sZXJhdGUgIAo+ID4gPiB3cml0 ZSBtZXJnaW5nLgo+ID4gPiAKPiA+ID4gIndyaXRlIG1lcmdpbmciICBUaGlzIGlzIGEgdmVyeSBz cGVjaWZpYyB0aGluZywgcGVyIFBDSSAzLjAsIDMuMi42Ogo+ID4gPiAKPiA+ID4gICBCeXRlIE1l cmdpbmcg4oCTIG9jY3VycyB3aGVuIGEgc2VxdWVuY2Ugb2YgaW5kaXZpZHVhbCBtZW1vcnkgd3Jp dGVzCj4gPiA+ICAgKGJ5dGVzIG9yIHdvcmRzKSBhcmUgbWVyZ2VkIGludG8gYSBzaW5nbGUgRFdP UkQuCj4gPiA+IAo+ID4gPiBUaGUgc2VtYW50aWNzIHN1Z2dlc3QgcXVhZHdvcmQgc3VwcG9ydCBp biBhZGRpdGlvbiB0byBkd29yZCwgYnV0Cj4gPiA+IGRvbid0IHJlcXVpcmUgaXQuICBXcml0ZXMg dG8gYnl0ZXMgd2l0aGluIGEgZHdvcmQgY2FuIGJlIG1lcmdlZCwKPiA+ID4gYnV0IGR1cGxpY2F0 ZSB3cml0ZXMgY2Fubm90Lgo+ID4gPiAKPiA+ID4gSXQgc2VlbXMgbGlrZSBhbiBleHRyZW1lbHkg bGliZXJhbCBhcHBsaWNhdGlvbiB0byBzdWdnZXN0IHRoYXQKPiA+ID4gdGhpcyBvbmUgd3JpdGUg c2VtYW50aWMgZW5jb21wYXNzZXMgZnVsbCB3cml0ZSBjb21iaW5pbmcKPiA+ID4gc2VtYW50aWNz LCB3aGljaCBpdHNlbGYgaXMgbm90IGNsZWFybHkgZGVmaW5lZC4KPiA+ID4gIAo+ID4gVGFsa2lu ZyB0byBvdXIgUENJZSBTSUcgcmVwcmVzZW50YXRpdmUsIFBDSWUgc3dpdGNoZXMgYXJlIG5vdAo+ ID4gYWxsb3dlZCBkbyBhbnkgb2YgdGhlIGJ5dGUgTWVyZ2luZy9jb21iaW5pbmcgZXRjIGFzIGRl ZmluZWQgaW4gdGhlCj4gPiBQQ0kgc3BlYywgYW5kIHBlciBhIHJhdGhlciBwb29ybHkgd29yZGVk IEltcGxlbWVudGF0aW9uIG5vdGUgaW4KPiA+IHRoZSBzcGVjIHNheXMgdGhhdCBubyBrbm93biBQ Q0llIEhvc3QgQnJpZGRnZXMvUm9vdCBwb3J0cyBkbyBpdAo+ID4gZWl0aGVyLiAgU28gZm9yIFBD SWUgd2UgZG9uJ3QgdGhpbmsgYmVsaWV2ZSB0aGVyZSBpcyBhbnkgYnl0ZQo+ID4gbWVyZ2luZyB0 aGF0IGhhcHBlbnMgaW4gdGhlIFBDSWUgZmFicmljIHNvIGl0J3MgcmVhbGx5IGEgbWF0dGVyIG9m Cj4gPiB3aGF0IGhhcHBlbnMgaW4gdGhlIENQVSBjb3JlIGFuZCBpbnRlcmNvbm5lY3QgYmVmb3Jl IGl0IGdldHMgdG8KPiA+IHRoZSBQQ0llIGhpZXJhcmNoeS4KPiAKPiBZZXMsIGJ1dCBtZXJnZWQg d3JpdGVzLCBubyBtYXR0ZXIgd2hlcmUgdGhleSBoYXBwZW4sIGFyZSBzdGlsbCB0aGUgb25seQo+ IHR5cGUgb2Ygd3JpdGUgY29tYmluaW5nIHRoYXQgYSBwcmVmZXRjaGFibGUgQkFSIG9uIGFuIGVu ZHBvaW50IGlzCj4gcmVxdWlyZWQgdG8gc3VwcG9ydC4KPiAKPiA+IFN0ZXBwaW5nIGJhY2sgZnJv bSB0aGlzIHBhdGNoc2V0LCBkbyB5b3UgYWdyZWUgdGhhdCBpdCBpcwo+ID4gZGVzaXJhYmxlIHRv IHN1cHBvcnQgV3JpdGUgY29tYmluaW5nIGFzIHVuZGVyc3Rvb2QgYnkgaW9yZW1hcF93Ywo+ID4g dG8gd29yayBpbiBhbGwgSVNBIGd1ZXN0cyBpbmNsdWRpbmcgQVJNdjg/Cj4gCj4gWWVzLCBhIHVz ZXJzcGFjZSB2ZmlvIGRyaXZlciBzaG91bGQgYmUgYWJsZSB0byB0YWtlIGFkdmFudGFnZSBvZiB0 aGUKPiBoYXJkd2FyZSBjYXBhYmlsaXRpZXMuICBJIHRoaW5rIHdoZXJlIHdlIGRpc2FncmVlIGlz IHdoZXRoZXIgaXQncwo+IHVuaXZlcnNhbGx5IHNhZmUgdG8gYXNzdW1lIHdyaXRlIGNvbWJpbmlu ZyBiYXNlZCBvbiB0aGUgUENJCj4gcHJlZmV0Y2hhYmxlIGNhcGFiaWxpdHkgb2YgYSBCQVIuICBJ ZiB0aGF0J3Mgc29tZXRoaW5nIHRoYXQgY2FuIGJlCj4gYXNzdW1lZCB1bml2ZXJzYWxseSBmb3Ig QVJNdjggYmFzZWQgb24gdGhlIGFyY2hpdGVjdHVyZSBzcGVjaWZpY2F0aW9uCj4gY29tcGF0aWJp bGl0eSB3aXRoIHRoZSBQQ0kgZGVmaW5pdGlvbiBvZiBhIHByZWZldGNoYWJsZSBCQVIsIHRoZW4g SQo+IHdvdWxkIGV4cGVjdCBhIGhlbHBlciBzb21ld2hlcmUgaW4gYXJjaCBjb2RlIHRoYXQgcmV0 dXJucyB0aGUgcmlnaHQKPiBwYWdlIHByb3RlY3Rpb24gZmxhZ3MsIHNvIHRoYXQgYXJjaCBtYWlu dGFpbmVycyBkb24ndCBuZWVkIHRvIHNjb3VyCj4gZGV2aWNlIGRyaXZlcnMgZm9yIGFyY2hpdGVj dHVyZSBoYWNrcy4gIE90aGVyd2lzZSwgaXQgbmVlZHMgdG8gYmUKPiBleHBvc2VkIHRocm91Z2gg dGhlIHZmaW8gdUFQSSB0byBhbGxvdyB0aGUgdXNlcnNwYWNlIGRldmljZSBkcml2ZXIKPiBpdHNl bGYgdG8gc2VsZWN0IHRoZXNlIHNlbWFudGljcy4KPiAKPiA+IFlvdSBub3RlIHRoYXQgeDg2IHZp cnR1YWxpemF0aW9uIGRvZXNuJ3QgaGF2ZSB0aGlzIGlzc3VlLCBidXQKPiA+IEtWTS1BUk0gZG9l cyBiZWNhdXNlIEtWTSBtYXBzIGFsbCBkZXZpY2UgQkFScyBhcyBEZXZpY2UgTWVtb3J5Cj4gPiB0 eXBlIG5HblJFIHdoaWNoIGRvZXNuJ3QgYWxsb3cgaW9yZW1hcF93YyBmcm9tIHdpdGhpbiB0aGUg Z3Vlc3QgdG8KPiA+IGdldCB0aGUgYWN0dWFsIHNlbWFudGljcyBkZXNpcmVkLgo+ID4gCj4gPiBN YXJjIGFuZCBvdGhlcnMgaGF2ZSBzdWdnZXN0ZWQgdGhhdCB1c2Vyc3BhY2Ugc2hvdWxkIHByb3Zp ZGUgdGhlCj4gPiBoaW50cy4gQnV0IHRoZSBxdWVzdGlvbiBpcyBob3cgd291bGQgcWVtdSB2Zmlv IGRvIHRoaXMgZWl0aGVyPyBXZQo+ID4gd291bGQgYmUgc3R1Y2sgaW4gdGhlIHNhbWUgYXJndW1l bnRzIGFzIGhlcmUsIGFzIHRvIHdoYXQgaXMgdGhlCj4gPiBjb3JyZWN0IHdheSB0byBkZXRlcm1p bmUgdGhlIGRlc2lyZWQgYXR0cmlidXRlcyBmb3IgYSBnaXZlbiBCQVIKPiA+IHN1Y2ggdGhhdCBl dmVudHVhbGx5IHdoZW4gYSBkcml2ZXIgaW4gdGhlIGd1ZXN0IGFza3MgZm9yCj4gPiBpb3JlbWFw X3djIGl0IGFjdHVhbGx5IGhhcyBhIGNoYW5jZSBvZiB3b3JraW5nIGluIHRoZSBndWVzdCwgaW4K PiA+IGFsbCBJU0FzLiAgRG8geW91IGhhdmUgYW55IHN1Z2dlc3Rpb25zIG9uIGhvdyB0byBtYWtl IHByb2dyZXNzCj4gPiBoZXJlPwo+IAo+IFdlIGRvIG5lZWQgc29tZSB3YXkgZm9yIHVzZXJzcGFj ZSBkcml2ZXJzIHRvIGFsc28gbWFrZSB1c2Ugb2YgV0MKPiBzZW1hbnRpY3MsIHRoZXJlIHdlcmUg c29tZSBkaXNjdXNzaW9ucyBpbiB0aGUgcGFzdCwgSSB0aGluayBvdGhlcnMgaGF2ZQo+IHJlZmVy ZW5jZWQgdGhlbSBhcyB3ZWxsLCBidXQgbm90aGluZyBoYXMgYmVlbiBwcm9wb3NlZCBmb3IgYSB2 ZmlvIEFQSS4KPiAKPiBJZiB3ZSBoYWQgdGhhdCBBUEksIFFFTVUgZGVjaWRpbmcgdG8gdW5pdmVy c2FsbHkgZW5hYmxlIFdDIGZvciBhbGwKPiB2ZmlvIHByZWZldGNoYWJsZSBCQVJzIHNlZW1zIG9u bHkgbWFyZ2luYWxseSBiZXR0ZXIgdGhhbiB0aGlzIGFwcHJvYWNoLgo+IFVsdGltYXRlbHkgdGhl IG1hcHBpbmcgc2hvdWxkIGJlIGJhc2VkIG9uIHRoZSBndWVzdCBkcml2ZXIgc2VtYW50aWNzLAo+ IGFuZCBpZiB5b3UgZG9uJ3QgaGF2ZSBhbnkgdmlzaWJpbGl0eSB0byB0aGF0IG9uIEtWTS9hcm0g bGlrZSB3ZSBoYXZlIG9uCj4gS1ZNL3g4NiwgdGhlbiBpdCBzZWVtcyBsaWtlIHRoZXJlJ3Mgbm90 aGluZyB0byB0cmlnZ2VyIGEgdmZpbyBBUEkgaGVyZQo+IGFueXdheS4KClRoZXJlIGlzbid0IG11 Y2ggS1ZNL2FybTY0IGNhbiBkbyBoZXJlIHVubGVzcyBpdCBpcyBiZWluZyB0b2xkIHdoYXQgdG8K ZG8uIFdlIGRvbid0IGhhdmUgdmlzaWJpbGl0eSBvbiB0aGUgZ3Vlc3QncyBwYWdlIHRhYmxlcyBp biBhIHJlbGlhYmxlCndheSwgYW5kIHRydXN0aW5nIHRoZW0gaXMgbm90IHNvbWV0aGluZyBJIHdh bnQgdG8gZW50ZXJ0YWluIGFueXdheS4KCj4gSWYgdGhhdCdzIHRoZSBjYXNlLCBJJ2QgcHJvYmFi bHkgZ28gYmFjayB0byBsZXR0aW5nIHRoZSBhcmNoL2FybTY0IGZvbGtzCj4gZGVjbGFyZSB0aGF0 IFdDIGlzIGNvbXBhdGlibGUgd2l0aCB0aGUgZGVmaW5pdGlvbiBvZiBQQ0kgcHJlZmV0Y2hhYmxl Cj4gYW5kIGV4cG9ydCBzb21lIHNvcnQgb2YgcGdwcm90X3BjaV9wcmVmZXRjaGFibGUoKSBoZWxw ZXIgd2hlcmUgdGhlCj4gZGVmYXVsdCB3b3VsZCBiZSB0byAjZGVmaW5lIGl0IGFzIHBncHJvY19u b25jYWNoZWQoKSAjaWZuZGVmIGJ5IHRoZQo+IGFyY2guCj4gCj4gPiBBIGRldmljZSBzcGVjaWZp YyBsaXN0IG9mIHdoaWNoIEJBUnMgYXJlIE9LIHRvIGFsbG93IGlvcmVtYXBfd2MKPiA+IGZvciBz ZWVtcyB0ZXJyaWJsZSBhbmQgSSdtIG5vdCBzdXJlIGlmIGEgY29tbWFuZGxpbmUgcWVtdSBvcHRp b24KPiA+IGlzIGFueSBiZXR0ZXIuIElzIHRoZSB1c2VyIG9mIGRldmljZSBhc3NpZ25tZW50L3N5 c2FkbWluIHN1cHBvc2VkCj4gPiB0byBrbm93IHdoaWNoIEJBUiBvZiB3aGljaCBkZXZpY2UgaXMg T0sgdG8gYWxsb3cgaW9yZW1hcF93YyBmb3I/Cj4gCj4gTm8sIGEgZGV2aWNlIHNwZWNpZmljIHVz ZXJzcGFjZSBkcml2ZXIgc2hvdWxkIGtub3cgc3VjaCBkZXZpY2UKPiBzZW1hbnRpY3MsIGJ1dCBR RU1VIGlzIG5vdCBzdWNoIGEgZHJpdmVyLiAgQnVyZGVuaW5nIHRoZSBoeXBlcnZpc29yCj4gdXNl ci9hZG1pbiBpcyBub3QgYSBnb29kIHNvbHV0aW9uIGVpdGhlci4gIEknZCBsZWFuIG9uIEtWTS9h cm02NCBmb2xrcwo+IHRvIGtub3cgaG93IHRoZSBndWVzdCBkcml2ZXIgc2VtYW50aWNzIGNhbiBi ZSBleHBvc2VkIHRvIHRoZQo+IGh5cGVydmlzb3IuICBUaGFua3MsCgpJIGRvbid0IHNlZSBhIGdv b2Qgd2F5IGZvciB0aGF0LCB1bmxlc3Mgd2UgbWFrZSBpdCBhIHBlci1ndWVzdCBidXktaW4Kd2hl cmUgYWxsIFBDSSBwcmVmZXRjaGFibGUgbWFwcGluZ3MgZ2V0IHRoZSBzYW1lIHRyZWF0bWVudC4g SSdtCnByZXBhcmVkIHRvIGJldCB0aGF0IHRoaXMgd2lsbCBicmVhayB3aGVuIHR3byBkZXZpY2Vz IHdpbGwgaGF2ZQpkaWZmZXJlbnQgcmVxdWlyZW1lbnRzLiBJdCB3b3VsZCBhbHNvIHJlcXVpcmUg dXNlcnNwYWNlIHRvIGJ1eSBpbnRvCnRoaXMgc2NoZW1lIHRob3VnaCwgd2hpY2ggaXMgY3JhcC4K CkV4cG9zaW5nIHRoZSBndWVzdCdzIHByZWZlcmVuY2Ugb24gYSBwZXItZGV2aWNlIGJhc2lzIHNl ZW1zIGRpZmZpY3VsdAooS1ZNIGtub3dzIG5vdGhpbmcgYWJvdXQgdGhlIFBDSSBkZXZpY2VzKSBh bmQgd291bGQgcmVxdWlyZSBzb21lIFBWCmludGVyZmFjZSB0aGF0IHdpbGwgcXVpY2tseSBiZWNv bWUgdW5tYWludGFpbmFibGUuCgoJTS4KCi0tIApXaXRob3V0IGRldmlhdGlvbiBmcm9tIHRoZSBu b3JtLCBwcm9ncmVzcyBpcyBub3QgcG9zc2libGUuCl9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fCmt2bWFybSBtYWlsaW5nIGxpc3QKa3ZtYXJtQGxpc3RzLmNz LmNvbHVtYmlhLmVkdQpodHRwczovL2xpc3RzLmNzLmNvbHVtYmlhLmVkdS9tYWlsbWFuL2xpc3Rp bmZvL2t2bWFybQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0F01C4708F for ; Wed, 2 Jun 2021 09:13:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8A6D2613AA for ; Wed, 2 Jun 2021 09:13:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A6D2613AA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=K3hsQrOElZi85cJkiKuSmriTfAWMJD5Wf9AbZRiEpOw=; b=VeOlNGFyBuCJMz s3SdrvopfxdVXcpeY+yphlW2BzCwuHLsDpexua06VGnoTxvoidid2YGeZ/OpAwLnlT21JwbhX/TI+ ETH83EusvpUs7ZCZ2dOkpgxNn7HhWXHz5/2LYKbuqGjH6Bjt2Hgs7sEAB2eGc/vjcXlwwTqyFQcrS mr0C2qz3syHOMakWuMgPoLlBxmv+6+n+HX5a3ypPyTtYPXCiecFnOdFSMhaBCyqwrzt6INT9X4W1Y k9rjX54GWY2O4oq1LdNLE18C1sqYOFYUHkM13VfxIezBNtSClOWSGRA20Uw7ih/skB7xPVa/1Xe8v 1d6+EIUkPjpK2uagqqbA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1loMuE-002tSJ-OE; Wed, 02 Jun 2021 09:11:19 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1loMu9-002tRc-3i for linux-arm-kernel@lists.infradead.org; Wed, 02 Jun 2021 09:11:14 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AD0FB608FE; Wed, 2 Jun 2021 09:11:12 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loMu6-004zLH-Ha; Wed, 02 Jun 2021 10:11:10 +0100 Date: Wed, 02 Jun 2021 10:11:09 +0100 Message-ID: <87a6o81viq.wl-maz@kernel.org> From: Marc Zyngier To: Alex Williamson Cc: Vikram Sethi , Mark Kettenis , Shanker Donthineni , "will@kernel.org" , "catalin.marinas@arm.com" , "christoffer.dall@arm.com" , "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Jason Sequeira Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA In-Reply-To: <20210504120348.2eec075b@redhat.com> References: <20210429162906.32742-1-sdonthineni@nvidia.com> <20210429162906.32742-2-sdonthineni@nvidia.com> <20210429122840.4f98f78e@redhat.com> <470360a7-0242-9ae5-816f-13608f957bf6@nvidia.com> <20210429134659.321a5c3c@redhat.com> <87czucngdc.wl-maz@kernel.org> <1edb2c4e-23f0-5730-245b-fc6d289951e1@nvidia.com> <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> <20210504120348.2eec075b@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: alex.williamson@redhat.com, vsethi@nvidia.com, mark.kettenis@xs4all.nl, sdonthineni@nvidia.com, will@kernel.org, catalin.marinas@arm.com, christoffer.dall@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, jsequeira@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210602_021113_237417_CEDC0E75 X-CRM114-Status: GOOD ( 65.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVHVlLCAwNCBNYXkgMjAyMSAxOTowMzo0OCArMDEwMCwKQWxleCBXaWxsaWFtc29uIDxhbGV4 LndpbGxpYW1zb25AcmVkaGF0LmNvbT4gd3JvdGU6Cj4gCj4gT24gTW9uLCAzIE1heSAyMDIxIDIy OjAzOjU5ICswMDAwCj4gVmlrcmFtIFNldGhpIDx2c2V0aGlAbnZpZGlhLmNvbT4gd3JvdGU6Cj4g Cj4gPiBIaSBBbGV4LAo+ID4gPiBGcm9tOiBBbGV4IFdpbGxpYW1zb24gPGFsZXgud2lsbGlhbXNv bkByZWRoYXQuY29tPgo+ID4gPiBPbiBNb24sIDMgTWF5IDIwMjEgMTM6NTk6NDMgKzAwMDAKPiA+ ID4gVmlrcmFtIFNldGhpIDx2c2V0aGlAbnZpZGlhLmNvbT4gd3JvdGU6ICAKPiA+ID4gPiA+IEZy b206IE1hcmsgS2V0dGVuaXMgPG1hcmsua2V0dGVuaXNAeHM0YWxsLm5sPiAgCj4gPiA+ID4gPiA+ IEZyb206IE1hcmMgWnluZ2llciA8bWF6QGtlcm5lbC5vcmc+ICAKPiA+ID4gPgo+ID4gPiA+IHNu aXAgIAo+ID4gPiA+ID4gPiBJZiwgYnkgZW51bWVyYXRpbmcgdGhlIHByb3BlcnRpZXMgb2YgUHJl ZmV0Y2hhYmxlLCB5b3UgY2FuIHNob3cKPiA+ID4gPiA+ID4gdGhhdCB0aGV5IGFyZSBhIHN0cmlj dCBzdXBlcnNldCBvZiBOb3JtYWxfTkMsIEknbSBvbiBib2FyZC4gSQo+ID4gPiA+ID4gPiBoYXZl bid0IHNlZW4gc3VjaCBhbiBlbnVtZXJhdGlvbiBzbyBmYXIuCj4gPiA+ID4gPiA+ICAKPiA+ID4g PiBzbmlwICAKPiA+ID4gPiA+ID4gUmlnaHQsIHNvIHdlIGhhdmUgbWFkZSBhIHNtYWxsIHN0ZXAg aW4gdGhlIGRpcmVjdGlvbiBvZiBtYXBwaW5nCj4gPiA+ID4gPiA+ICJwcmVmZXRjaGFibGUiIG9u dG8gIk5vcm1hbF9OQyIsIHRoYW5rcyBmb3IgdGhhdC4gV2hhdCBhYm91dCBhbGwKPiA+ID4gPiA+ ID4gdGhlIG90aGVyIHByb3BlcnRpZXMgKHVuYWxpZ25lZCBhY2Nlc3Nlcywgb3JkZXJpbmcsIGdh dGhlcmluZyk/ICAKPiA+ID4gPiA+ICAKPiA+ID4gPiBSZWdhcmRpbmcgZ2F0aGVyaW5nL3dyaXRl IGNvbWJpbmluZywgdGhhdCBpcyBhbHNvIGFsbG93ZWQgdG8KPiA+ID4gPiBwcmVmZXRjaGFibGUg cGVyIFBDSSBzcGVjICAKPiA+ID4gCj4gPiA+IEFzIG90aGVycyBoYXZlIHN0YXRlZCwgZ2F0aGVy L3dyaXRlIGNvbWJpbmluZyBpdHNlbGYgaXMgbm90IHdlbGwgZGVmaW5lZC4KPiA+ID4gICAKPiA+ ID4gPiBGcm9tIDEuMy4yLjIgb2YgNS8wIGJhc2Ugc3BlYzoKPiA+ID4gPiBBIFBDSSBFeHByZXNz IEVuZHBvaW50IHJlcXVlc3RpbmcgbWVtb3J5IHJlc291cmNlcyB0aHJvdWdoIGEgQkFSIG11c3QK PiA+ID4gPiBzZXQgdGhlIEJBUidzIFByZWZldGNoYWJsZSBiaXQgdW5sZXNzIHRoZSByYW5nZSBj b250YWlucyBsb2NhdGlvbnMKPiA+ID4gPiB3aXRoIHJlYWQgc2lkZS1lZmZlY3RzIG9yIGxvY2F0 aW9ucyBpbiB3aGljaCB0aGUgRnVuY3Rpb24gZG9lcyBub3QgdG9sZXJhdGUgIAo+ID4gPiB3cml0 ZSBtZXJnaW5nLgo+ID4gPiAKPiA+ID4gIndyaXRlIG1lcmdpbmciICBUaGlzIGlzIGEgdmVyeSBz cGVjaWZpYyB0aGluZywgcGVyIFBDSSAzLjAsIDMuMi42Ogo+ID4gPiAKPiA+ID4gICBCeXRlIE1l cmdpbmcg4oCTIG9jY3VycyB3aGVuIGEgc2VxdWVuY2Ugb2YgaW5kaXZpZHVhbCBtZW1vcnkgd3Jp dGVzCj4gPiA+ICAgKGJ5dGVzIG9yIHdvcmRzKSBhcmUgbWVyZ2VkIGludG8gYSBzaW5nbGUgRFdP UkQuCj4gPiA+IAo+ID4gPiBUaGUgc2VtYW50aWNzIHN1Z2dlc3QgcXVhZHdvcmQgc3VwcG9ydCBp biBhZGRpdGlvbiB0byBkd29yZCwgYnV0Cj4gPiA+IGRvbid0IHJlcXVpcmUgaXQuICBXcml0ZXMg dG8gYnl0ZXMgd2l0aGluIGEgZHdvcmQgY2FuIGJlIG1lcmdlZCwKPiA+ID4gYnV0IGR1cGxpY2F0 ZSB3cml0ZXMgY2Fubm90Lgo+ID4gPiAKPiA+ID4gSXQgc2VlbXMgbGlrZSBhbiBleHRyZW1lbHkg bGliZXJhbCBhcHBsaWNhdGlvbiB0byBzdWdnZXN0IHRoYXQKPiA+ID4gdGhpcyBvbmUgd3JpdGUg c2VtYW50aWMgZW5jb21wYXNzZXMgZnVsbCB3cml0ZSBjb21iaW5pbmcKPiA+ID4gc2VtYW50aWNz LCB3aGljaCBpdHNlbGYgaXMgbm90IGNsZWFybHkgZGVmaW5lZC4KPiA+ID4gIAo+ID4gVGFsa2lu ZyB0byBvdXIgUENJZSBTSUcgcmVwcmVzZW50YXRpdmUsIFBDSWUgc3dpdGNoZXMgYXJlIG5vdAo+ ID4gYWxsb3dlZCBkbyBhbnkgb2YgdGhlIGJ5dGUgTWVyZ2luZy9jb21iaW5pbmcgZXRjIGFzIGRl ZmluZWQgaW4gdGhlCj4gPiBQQ0kgc3BlYywgYW5kIHBlciBhIHJhdGhlciBwb29ybHkgd29yZGVk IEltcGxlbWVudGF0aW9uIG5vdGUgaW4KPiA+IHRoZSBzcGVjIHNheXMgdGhhdCBubyBrbm93biBQ Q0llIEhvc3QgQnJpZGRnZXMvUm9vdCBwb3J0cyBkbyBpdAo+ID4gZWl0aGVyLiAgU28gZm9yIFBD SWUgd2UgZG9uJ3QgdGhpbmsgYmVsaWV2ZSB0aGVyZSBpcyBhbnkgYnl0ZQo+ID4gbWVyZ2luZyB0 aGF0IGhhcHBlbnMgaW4gdGhlIFBDSWUgZmFicmljIHNvIGl0J3MgcmVhbGx5IGEgbWF0dGVyIG9m Cj4gPiB3aGF0IGhhcHBlbnMgaW4gdGhlIENQVSBjb3JlIGFuZCBpbnRlcmNvbm5lY3QgYmVmb3Jl IGl0IGdldHMgdG8KPiA+IHRoZSBQQ0llIGhpZXJhcmNoeS4KPiAKPiBZZXMsIGJ1dCBtZXJnZWQg d3JpdGVzLCBubyBtYXR0ZXIgd2hlcmUgdGhleSBoYXBwZW4sIGFyZSBzdGlsbCB0aGUgb25seQo+ IHR5cGUgb2Ygd3JpdGUgY29tYmluaW5nIHRoYXQgYSBwcmVmZXRjaGFibGUgQkFSIG9uIGFuIGVu ZHBvaW50IGlzCj4gcmVxdWlyZWQgdG8gc3VwcG9ydC4KPiAKPiA+IFN0ZXBwaW5nIGJhY2sgZnJv bSB0aGlzIHBhdGNoc2V0LCBkbyB5b3UgYWdyZWUgdGhhdCBpdCBpcwo+ID4gZGVzaXJhYmxlIHRv IHN1cHBvcnQgV3JpdGUgY29tYmluaW5nIGFzIHVuZGVyc3Rvb2QgYnkgaW9yZW1hcF93Ywo+ID4g dG8gd29yayBpbiBhbGwgSVNBIGd1ZXN0cyBpbmNsdWRpbmcgQVJNdjg/Cj4gCj4gWWVzLCBhIHVz ZXJzcGFjZSB2ZmlvIGRyaXZlciBzaG91bGQgYmUgYWJsZSB0byB0YWtlIGFkdmFudGFnZSBvZiB0 aGUKPiBoYXJkd2FyZSBjYXBhYmlsaXRpZXMuICBJIHRoaW5rIHdoZXJlIHdlIGRpc2FncmVlIGlz IHdoZXRoZXIgaXQncwo+IHVuaXZlcnNhbGx5IHNhZmUgdG8gYXNzdW1lIHdyaXRlIGNvbWJpbmlu ZyBiYXNlZCBvbiB0aGUgUENJCj4gcHJlZmV0Y2hhYmxlIGNhcGFiaWxpdHkgb2YgYSBCQVIuICBJ ZiB0aGF0J3Mgc29tZXRoaW5nIHRoYXQgY2FuIGJlCj4gYXNzdW1lZCB1bml2ZXJzYWxseSBmb3Ig QVJNdjggYmFzZWQgb24gdGhlIGFyY2hpdGVjdHVyZSBzcGVjaWZpY2F0aW9uCj4gY29tcGF0aWJp bGl0eSB3aXRoIHRoZSBQQ0kgZGVmaW5pdGlvbiBvZiBhIHByZWZldGNoYWJsZSBCQVIsIHRoZW4g SQo+IHdvdWxkIGV4cGVjdCBhIGhlbHBlciBzb21ld2hlcmUgaW4gYXJjaCBjb2RlIHRoYXQgcmV0 dXJucyB0aGUgcmlnaHQKPiBwYWdlIHByb3RlY3Rpb24gZmxhZ3MsIHNvIHRoYXQgYXJjaCBtYWlu dGFpbmVycyBkb24ndCBuZWVkIHRvIHNjb3VyCj4gZGV2aWNlIGRyaXZlcnMgZm9yIGFyY2hpdGVj dHVyZSBoYWNrcy4gIE90aGVyd2lzZSwgaXQgbmVlZHMgdG8gYmUKPiBleHBvc2VkIHRocm91Z2gg dGhlIHZmaW8gdUFQSSB0byBhbGxvdyB0aGUgdXNlcnNwYWNlIGRldmljZSBkcml2ZXIKPiBpdHNl bGYgdG8gc2VsZWN0IHRoZXNlIHNlbWFudGljcy4KPiAKPiA+IFlvdSBub3RlIHRoYXQgeDg2IHZp cnR1YWxpemF0aW9uIGRvZXNuJ3QgaGF2ZSB0aGlzIGlzc3VlLCBidXQKPiA+IEtWTS1BUk0gZG9l cyBiZWNhdXNlIEtWTSBtYXBzIGFsbCBkZXZpY2UgQkFScyBhcyBEZXZpY2UgTWVtb3J5Cj4gPiB0 eXBlIG5HblJFIHdoaWNoIGRvZXNuJ3QgYWxsb3cgaW9yZW1hcF93YyBmcm9tIHdpdGhpbiB0aGUg Z3Vlc3QgdG8KPiA+IGdldCB0aGUgYWN0dWFsIHNlbWFudGljcyBkZXNpcmVkLgo+ID4gCj4gPiBN YXJjIGFuZCBvdGhlcnMgaGF2ZSBzdWdnZXN0ZWQgdGhhdCB1c2Vyc3BhY2Ugc2hvdWxkIHByb3Zp ZGUgdGhlCj4gPiBoaW50cy4gQnV0IHRoZSBxdWVzdGlvbiBpcyBob3cgd291bGQgcWVtdSB2Zmlv IGRvIHRoaXMgZWl0aGVyPyBXZQo+ID4gd291bGQgYmUgc3R1Y2sgaW4gdGhlIHNhbWUgYXJndW1l bnRzIGFzIGhlcmUsIGFzIHRvIHdoYXQgaXMgdGhlCj4gPiBjb3JyZWN0IHdheSB0byBkZXRlcm1p bmUgdGhlIGRlc2lyZWQgYXR0cmlidXRlcyBmb3IgYSBnaXZlbiBCQVIKPiA+IHN1Y2ggdGhhdCBl dmVudHVhbGx5IHdoZW4gYSBkcml2ZXIgaW4gdGhlIGd1ZXN0IGFza3MgZm9yCj4gPiBpb3JlbWFw X3djIGl0IGFjdHVhbGx5IGhhcyBhIGNoYW5jZSBvZiB3b3JraW5nIGluIHRoZSBndWVzdCwgaW4K PiA+IGFsbCBJU0FzLiAgRG8geW91IGhhdmUgYW55IHN1Z2dlc3Rpb25zIG9uIGhvdyB0byBtYWtl IHByb2dyZXNzCj4gPiBoZXJlPwo+IAo+IFdlIGRvIG5lZWQgc29tZSB3YXkgZm9yIHVzZXJzcGFj ZSBkcml2ZXJzIHRvIGFsc28gbWFrZSB1c2Ugb2YgV0MKPiBzZW1hbnRpY3MsIHRoZXJlIHdlcmUg c29tZSBkaXNjdXNzaW9ucyBpbiB0aGUgcGFzdCwgSSB0aGluayBvdGhlcnMgaGF2ZQo+IHJlZmVy ZW5jZWQgdGhlbSBhcyB3ZWxsLCBidXQgbm90aGluZyBoYXMgYmVlbiBwcm9wb3NlZCBmb3IgYSB2 ZmlvIEFQSS4KPiAKPiBJZiB3ZSBoYWQgdGhhdCBBUEksIFFFTVUgZGVjaWRpbmcgdG8gdW5pdmVy c2FsbHkgZW5hYmxlIFdDIGZvciBhbGwKPiB2ZmlvIHByZWZldGNoYWJsZSBCQVJzIHNlZW1zIG9u bHkgbWFyZ2luYWxseSBiZXR0ZXIgdGhhbiB0aGlzIGFwcHJvYWNoLgo+IFVsdGltYXRlbHkgdGhl IG1hcHBpbmcgc2hvdWxkIGJlIGJhc2VkIG9uIHRoZSBndWVzdCBkcml2ZXIgc2VtYW50aWNzLAo+ IGFuZCBpZiB5b3UgZG9uJ3QgaGF2ZSBhbnkgdmlzaWJpbGl0eSB0byB0aGF0IG9uIEtWTS9hcm0g bGlrZSB3ZSBoYXZlIG9uCj4gS1ZNL3g4NiwgdGhlbiBpdCBzZWVtcyBsaWtlIHRoZXJlJ3Mgbm90 aGluZyB0byB0cmlnZ2VyIGEgdmZpbyBBUEkgaGVyZQo+IGFueXdheS4KClRoZXJlIGlzbid0IG11 Y2ggS1ZNL2FybTY0IGNhbiBkbyBoZXJlIHVubGVzcyBpdCBpcyBiZWluZyB0b2xkIHdoYXQgdG8K ZG8uIFdlIGRvbid0IGhhdmUgdmlzaWJpbGl0eSBvbiB0aGUgZ3Vlc3QncyBwYWdlIHRhYmxlcyBp biBhIHJlbGlhYmxlCndheSwgYW5kIHRydXN0aW5nIHRoZW0gaXMgbm90IHNvbWV0aGluZyBJIHdh bnQgdG8gZW50ZXJ0YWluIGFueXdheS4KCj4gSWYgdGhhdCdzIHRoZSBjYXNlLCBJJ2QgcHJvYmFi bHkgZ28gYmFjayB0byBsZXR0aW5nIHRoZSBhcmNoL2FybTY0IGZvbGtzCj4gZGVjbGFyZSB0aGF0 IFdDIGlzIGNvbXBhdGlibGUgd2l0aCB0aGUgZGVmaW5pdGlvbiBvZiBQQ0kgcHJlZmV0Y2hhYmxl Cj4gYW5kIGV4cG9ydCBzb21lIHNvcnQgb2YgcGdwcm90X3BjaV9wcmVmZXRjaGFibGUoKSBoZWxw ZXIgd2hlcmUgdGhlCj4gZGVmYXVsdCB3b3VsZCBiZSB0byAjZGVmaW5lIGl0IGFzIHBncHJvY19u b25jYWNoZWQoKSAjaWZuZGVmIGJ5IHRoZQo+IGFyY2guCj4gCj4gPiBBIGRldmljZSBzcGVjaWZp YyBsaXN0IG9mIHdoaWNoIEJBUnMgYXJlIE9LIHRvIGFsbG93IGlvcmVtYXBfd2MKPiA+IGZvciBz ZWVtcyB0ZXJyaWJsZSBhbmQgSSdtIG5vdCBzdXJlIGlmIGEgY29tbWFuZGxpbmUgcWVtdSBvcHRp b24KPiA+IGlzIGFueSBiZXR0ZXIuIElzIHRoZSB1c2VyIG9mIGRldmljZSBhc3NpZ25tZW50L3N5 c2FkbWluIHN1cHBvc2VkCj4gPiB0byBrbm93IHdoaWNoIEJBUiBvZiB3aGljaCBkZXZpY2UgaXMg T0sgdG8gYWxsb3cgaW9yZW1hcF93YyBmb3I/Cj4gCj4gTm8sIGEgZGV2aWNlIHNwZWNpZmljIHVz ZXJzcGFjZSBkcml2ZXIgc2hvdWxkIGtub3cgc3VjaCBkZXZpY2UKPiBzZW1hbnRpY3MsIGJ1dCBR RU1VIGlzIG5vdCBzdWNoIGEgZHJpdmVyLiAgQnVyZGVuaW5nIHRoZSBoeXBlcnZpc29yCj4gdXNl ci9hZG1pbiBpcyBub3QgYSBnb29kIHNvbHV0aW9uIGVpdGhlci4gIEknZCBsZWFuIG9uIEtWTS9h cm02NCBmb2xrcwo+IHRvIGtub3cgaG93IHRoZSBndWVzdCBkcml2ZXIgc2VtYW50aWNzIGNhbiBi ZSBleHBvc2VkIHRvIHRoZQo+IGh5cGVydmlzb3IuICBUaGFua3MsCgpJIGRvbid0IHNlZSBhIGdv b2Qgd2F5IGZvciB0aGF0LCB1bmxlc3Mgd2UgbWFrZSBpdCBhIHBlci1ndWVzdCBidXktaW4Kd2hl cmUgYWxsIFBDSSBwcmVmZXRjaGFibGUgbWFwcGluZ3MgZ2V0IHRoZSBzYW1lIHRyZWF0bWVudC4g SSdtCnByZXBhcmVkIHRvIGJldCB0aGF0IHRoaXMgd2lsbCBicmVhayB3aGVuIHR3byBkZXZpY2Vz IHdpbGwgaGF2ZQpkaWZmZXJlbnQgcmVxdWlyZW1lbnRzLiBJdCB3b3VsZCBhbHNvIHJlcXVpcmUg dXNlcnNwYWNlIHRvIGJ1eSBpbnRvCnRoaXMgc2NoZW1lIHRob3VnaCwgd2hpY2ggaXMgY3JhcC4K CkV4cG9zaW5nIHRoZSBndWVzdCdzIHByZWZlcmVuY2Ugb24gYSBwZXItZGV2aWNlIGJhc2lzIHNl ZW1zIGRpZmZpY3VsdAooS1ZNIGtub3dzIG5vdGhpbmcgYWJvdXQgdGhlIFBDSSBkZXZpY2VzKSBh bmQgd291bGQgcmVxdWlyZSBzb21lIFBWCmludGVyZmFjZSB0aGF0IHdpbGwgcXVpY2tseSBiZWNv bWUgdW5tYWludGFpbmFibGUuCgoJTS4KCi0tIApXaXRob3V0IGRldmlhdGlvbiBmcm9tIHRoZSBu b3JtLCBwcm9ncmVzcyBpcyBub3QgcG9zc2libGUuCgpfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51 eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5v cmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD3C5C4708F for ; Wed, 2 Jun 2021 09:11:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B962361003 for ; Wed, 2 Jun 2021 09:11:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230193AbhFBJM4 convert rfc822-to-8bit (ORCPT ); Wed, 2 Jun 2021 05:12:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:46526 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229745AbhFBJMz (ORCPT ); Wed, 2 Jun 2021 05:12:55 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AD0FB608FE; Wed, 2 Jun 2021 09:11:12 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loMu6-004zLH-Ha; Wed, 02 Jun 2021 10:11:10 +0100 Date: Wed, 02 Jun 2021 10:11:09 +0100 Message-ID: <87a6o81viq.wl-maz@kernel.org> From: Marc Zyngier To: Alex Williamson Cc: Vikram Sethi , Mark Kettenis , Shanker Donthineni , "will@kernel.org" , "catalin.marinas@arm.com" , "christoffer.dall@arm.com" , "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Jason Sequeira Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA In-Reply-To: <20210504120348.2eec075b@redhat.com> References: <20210429162906.32742-1-sdonthineni@nvidia.com> <20210429162906.32742-2-sdonthineni@nvidia.com> <20210429122840.4f98f78e@redhat.com> <470360a7-0242-9ae5-816f-13608f957bf6@nvidia.com> <20210429134659.321a5c3c@redhat.com> <87czucngdc.wl-maz@kernel.org> <1edb2c4e-23f0-5730-245b-fc6d289951e1@nvidia.com> <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> <20210504120348.2eec075b@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: alex.williamson@redhat.com, vsethi@nvidia.com, mark.kettenis@xs4all.nl, sdonthineni@nvidia.com, will@kernel.org, catalin.marinas@arm.com, christoffer.dall@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, jsequeira@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Tue, 04 May 2021 19:03:48 +0100, Alex Williamson wrote: > > On Mon, 3 May 2021 22:03:59 +0000 > Vikram Sethi wrote: > > > Hi Alex, > > > From: Alex Williamson > > > On Mon, 3 May 2021 13:59:43 +0000 > > > Vikram Sethi wrote: > > > > > From: Mark Kettenis > > > > > > From: Marc Zyngier > > > > > > > > snip > > > > > > If, by enumerating the properties of Prefetchable, you can show > > > > > > that they are a strict superset of Normal_NC, I'm on board. I > > > > > > haven't seen such an enumeration so far. > > > > > > > > > > snip > > > > > > Right, so we have made a small step in the direction of mapping > > > > > > "prefetchable" onto "Normal_NC", thanks for that. What about all > > > > > > the other properties (unaligned accesses, ordering, gathering)? > > > > > > > > > Regarding gathering/write combining, that is also allowed to > > > > prefetchable per PCI spec > > > > > > As others have stated, gather/write combining itself is not well defined. > > > > > > > From 1.3.2.2 of 5/0 base spec: > > > > A PCI Express Endpoint requesting memory resources through a BAR must > > > > set the BAR's Prefetchable bit unless the range contains locations > > > > with read side-effects or locations in which the Function does not tolerate > > > write merging. > > > > > > "write merging" This is a very specific thing, per PCI 3.0, 3.2.6: > > > > > > Byte Merging – occurs when a sequence of individual memory writes > > > (bytes or words) are merged into a single DWORD. > > > > > > The semantics suggest quadword support in addition to dword, but > > > don't require it. Writes to bytes within a dword can be merged, > > > but duplicate writes cannot. > > > > > > It seems like an extremely liberal application to suggest that > > > this one write semantic encompasses full write combining > > > semantics, which itself is not clearly defined. > > > > > Talking to our PCIe SIG representative, PCIe switches are not > > allowed do any of the byte Merging/combining etc as defined in the > > PCI spec, and per a rather poorly worded Implementation note in > > the spec says that no known PCIe Host Briddges/Root ports do it > > either. So for PCIe we don't think believe there is any byte > > merging that happens in the PCIe fabric so it's really a matter of > > what happens in the CPU core and interconnect before it gets to > > the PCIe hierarchy. > > Yes, but merged writes, no matter where they happen, are still the only > type of write combining that a prefetchable BAR on an endpoint is > required to support. > > > Stepping back from this patchset, do you agree that it is > > desirable to support Write combining as understood by ioremap_wc > > to work in all ISA guests including ARMv8? > > Yes, a userspace vfio driver should be able to take advantage of the > hardware capabilities. I think where we disagree is whether it's > universally safe to assume write combining based on the PCI > prefetchable capability of a BAR. If that's something that can be > assumed universally for ARMv8 based on the architecture specification > compatibility with the PCI definition of a prefetchable BAR, then I > would expect a helper somewhere in arch code that returns the right > page protection flags, so that arch maintainers don't need to scour > device drivers for architecture hacks. Otherwise, it needs to be > exposed through the vfio uAPI to allow the userspace device driver > itself to select these semantics. > > > You note that x86 virtualization doesn't have this issue, but > > KVM-ARM does because KVM maps all device BARs as Device Memory > > type nGnRE which doesn't allow ioremap_wc from within the guest to > > get the actual semantics desired. > > > > Marc and others have suggested that userspace should provide the > > hints. But the question is how would qemu vfio do this either? We > > would be stuck in the same arguments as here, as to what is the > > correct way to determine the desired attributes for a given BAR > > such that eventually when a driver in the guest asks for > > ioremap_wc it actually has a chance of working in the guest, in > > all ISAs. Do you have any suggestions on how to make progress > > here? > > We do need some way for userspace drivers to also make use of WC > semantics, there were some discussions in the past, I think others have > referenced them as well, but nothing has been proposed for a vfio API. > > If we had that API, QEMU deciding to universally enable WC for all > vfio prefetchable BARs seems only marginally better than this approach. > Ultimately the mapping should be based on the guest driver semantics, > and if you don't have any visibility to that on KVM/arm like we have on > KVM/x86, then it seems like there's nothing to trigger a vfio API here > anyway. There isn't much KVM/arm64 can do here unless it is being told what to do. We don't have visibility on the guest's page tables in a reliable way, and trusting them is not something I want to entertain anyway. > If that's the case, I'd probably go back to letting the arch/arm64 folks > declare that WC is compatible with the definition of PCI prefetchable > and export some sort of pgprot_pci_prefetchable() helper where the > default would be to #define it as pgproc_noncached() #ifndef by the > arch. > > > A device specific list of which BARs are OK to allow ioremap_wc > > for seems terrible and I'm not sure if a commandline qemu option > > is any better. Is the user of device assignment/sysadmin supposed > > to know which BAR of which device is OK to allow ioremap_wc for? > > No, a device specific userspace driver should know such device > semantics, but QEMU is not such a driver. Burdening the hypervisor > user/admin is not a good solution either. I'd lean on KVM/arm64 folks > to know how the guest driver semantics can be exposed to the > hypervisor. Thanks, I don't see a good way for that, unless we make it a per-guest buy-in where all PCI prefetchable mappings get the same treatment. I'm prepared to bet that this will break when two devices will have different requirements. It would also require userspace to buy into this scheme though, which is crap. Exposing the guest's preference on a per-device basis seems difficult (KVM knows nothing about the PCI devices) and would require some PV interface that will quickly become unmaintainable. M. -- Without deviation from the norm, progress is not possible.