From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 819C0C43461 for ; Tue, 4 May 2021 18:03:59 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 001E7610EA for ; Tue, 4 May 2021 18:03:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 001E7610EA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 6B3ED4B4E8; Tue, 4 May 2021 14:03:58 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@redhat.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BsqR2RZ+m9k7; Tue, 4 May 2021 14:03:57 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 104EB4B380; Tue, 4 May 2021 14:03:57 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 7ED3C4B380 for ; Tue, 4 May 2021 14:03:55 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qwLchIHKuUhS for ; Tue, 4 May 2021 14:03:54 -0400 (EDT) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 5A3C84B2E2 for ; Tue, 4 May 2021 14:03:54 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620151434; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=crIjxSKc9RicXz5uU0iWJNjyfJ75605vwRv4rA88bAc=; b=gpHXQycL4j1chUFogkUq3N3XYM51I0LG5ZdwWX48Dko7I3NabRp6/taOl6StPF0HX8x6C5 XSH7sSZm7qvFLMfppTaOkjF2+mlpSe6pyKJrVzhb5u8HomiNge0XTT1ZE5B7Dir4tc6MKo O8Gr3kxW2y6wLmNaY0/qiEoM7lQJuWY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-538-UeGCIxtyO5OSuVhwk3SZ8A-1; Tue, 04 May 2021 14:03:51 -0400 X-MC-Unique: UeGCIxtyO5OSuVhwk3SZ8A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BE4AA107ACED; Tue, 4 May 2021 18:03:49 +0000 (UTC) Received: from redhat.com (ovpn-113-225.phx2.redhat.com [10.3.113.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F9921002388; Tue, 4 May 2021 18:03:49 +0000 (UTC) Date: Tue, 4 May 2021 12:03:48 -0600 From: Alex Williamson To: Vikram Sethi Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA Message-ID: <20210504120348.2eec075b@redhat.com> In-Reply-To: References: <20210429162906.32742-1-sdonthineni@nvidia.com> <20210429162906.32742-2-sdonthineni@nvidia.com> <20210429122840.4f98f78e@redhat.com> <470360a7-0242-9ae5-816f-13608f957bf6@nvidia.com> <20210429134659.321a5c3c@redhat.com> <87czucngdc.wl-maz@kernel.org> <1edb2c4e-23f0-5730-245b-fc6d289951e1@nvidia.com> <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Cc: "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Marc Zyngier , Shanker Donthineni , "catalin.marinas@arm.com" , Jason Sequeira , "will@kernel.org" , "kvmarm@lists.cs.columbia.edu" , "linux-arm-kernel@lists.infradead.org" , Mark Kettenis X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu T24gTW9uLCAzIE1heSAyMDIxIDIyOjAzOjU5ICswMDAwClZpa3JhbSBTZXRoaSA8dnNldGhpQG52 aWRpYS5jb20+IHdyb3RlOgoKPiBIaSBBbGV4LAo+ID4gRnJvbTogQWxleCBXaWxsaWFtc29uIDxh bGV4LndpbGxpYW1zb25AcmVkaGF0LmNvbT4KPiA+IE9uIE1vbiwgMyBNYXkgMjAyMSAxMzo1OTo0 MyArMDAwMAo+ID4gVmlrcmFtIFNldGhpIDx2c2V0aGlAbnZpZGlhLmNvbT4gd3JvdGU6ICAKPiA+ ID4gPiBGcm9tOiBNYXJrIEtldHRlbmlzIDxtYXJrLmtldHRlbmlzQHhzNGFsbC5ubD4gIAo+ID4g PiA+ID4gRnJvbTogTWFyYyBaeW5naWVyIDxtYXpAa2VybmVsLm9yZz4gIAo+ID4gPgo+ID4gPiBz bmlwICAKPiA+ID4gPiA+IElmLCBieSBlbnVtZXJhdGluZyB0aGUgcHJvcGVydGllcyBvZiBQcmVm ZXRjaGFibGUsIHlvdSBjYW4gc2hvdwo+ID4gPiA+ID4gdGhhdCB0aGV5IGFyZSBhIHN0cmljdCBz dXBlcnNldCBvZiBOb3JtYWxfTkMsIEknbSBvbiBib2FyZC4gSQo+ID4gPiA+ID4gaGF2ZW4ndCBz ZWVuIHN1Y2ggYW4gZW51bWVyYXRpb24gc28gZmFyLgo+ID4gPiA+ID4gIAo+ID4gPiBzbmlwICAK PiA+ID4gPiA+IFJpZ2h0LCBzbyB3ZSBoYXZlIG1hZGUgYSBzbWFsbCBzdGVwIGluIHRoZSBkaXJl Y3Rpb24gb2YgbWFwcGluZwo+ID4gPiA+ID4gInByZWZldGNoYWJsZSIgb250byAiTm9ybWFsX05D IiwgdGhhbmtzIGZvciB0aGF0LiBXaGF0IGFib3V0IGFsbAo+ID4gPiA+ID4gdGhlIG90aGVyIHBy b3BlcnRpZXMgKHVuYWxpZ25lZCBhY2Nlc3Nlcywgb3JkZXJpbmcsIGdhdGhlcmluZyk/ICAKPiA+ ID4gPiAgCj4gPiA+IFJlZ2FyZGluZyBnYXRoZXJpbmcvd3JpdGUgY29tYmluaW5nLCB0aGF0IGlz IGFsc28gYWxsb3dlZCB0bwo+ID4gPiBwcmVmZXRjaGFibGUgcGVyIFBDSSBzcGVjICAKPiA+IAo+ ID4gQXMgb3RoZXJzIGhhdmUgc3RhdGVkLCBnYXRoZXIvd3JpdGUgY29tYmluaW5nIGl0c2VsZiBp cyBub3Qgd2VsbCBkZWZpbmVkLgo+ID4gICAKPiA+ID4gRnJvbSAxLjMuMi4yIG9mIDUvMCBiYXNl IHNwZWM6Cj4gPiA+IEEgUENJIEV4cHJlc3MgRW5kcG9pbnQgcmVxdWVzdGluZyBtZW1vcnkgcmVz b3VyY2VzIHRocm91Z2ggYSBCQVIgbXVzdAo+ID4gPiBzZXQgdGhlIEJBUidzIFByZWZldGNoYWJs ZSBiaXQgdW5sZXNzIHRoZSByYW5nZSBjb250YWlucyBsb2NhdGlvbnMKPiA+ID4gd2l0aCByZWFk IHNpZGUtZWZmZWN0cyBvciBsb2NhdGlvbnMgaW4gd2hpY2ggdGhlIEZ1bmN0aW9uIGRvZXMgbm90 IHRvbGVyYXRlICAKPiA+IHdyaXRlIG1lcmdpbmcuCj4gPiAKPiA+ICJ3cml0ZSBtZXJnaW5nIiAg VGhpcyBpcyBhIHZlcnkgc3BlY2lmaWMgdGhpbmcsIHBlciBQQ0kgMy4wLCAzLjIuNjoKPiA+IAo+ ID4gICBCeXRlIE1lcmdpbmcg4oCTIG9jY3VycyB3aGVuIGEgc2VxdWVuY2Ugb2YgaW5kaXZpZHVh bCBtZW1vcnkgd3JpdGVzCj4gPiAgIChieXRlcyBvciB3b3JkcykgYXJlIG1lcmdlZCBpbnRvIGEg c2luZ2xlIERXT1JELgo+ID4gCj4gPiBUaGUgc2VtYW50aWNzIHN1Z2dlc3QgcXVhZHdvcmQgc3Vw cG9ydCBpbiBhZGRpdGlvbiB0byBkd29yZCwgYnV0IGRvbid0Cj4gPiByZXF1aXJlIGl0LiAgV3Jp dGVzIHRvIGJ5dGVzIHdpdGhpbiBhIGR3b3JkIGNhbiBiZSBtZXJnZWQsIGJ1dCBkdXBsaWNhdGUK PiA+IHdyaXRlcyBjYW5ub3QuCj4gPiAKPiA+IEl0IHNlZW1zIGxpa2UgYW4gZXh0cmVtZWx5IGxp YmVyYWwgYXBwbGljYXRpb24gdG8gc3VnZ2VzdCB0aGF0IHRoaXMgb25lIHdyaXRlCj4gPiBzZW1h bnRpYyBlbmNvbXBhc3NlcyBmdWxsIHdyaXRlIGNvbWJpbmluZyBzZW1hbnRpY3MsIHdoaWNoIGl0 c2VsZiBpcyBub3QKPiA+IGNsZWFybHkgZGVmaW5lZC4KPiA+ICAKPiBUYWxraW5nIHRvIG91ciBQ Q0llIFNJRyByZXByZXNlbnRhdGl2ZSwgUENJZSBzd2l0Y2hlcyBhcmUgbm90IGFsbG93ZWQgZG8g YW55IG9mIHRoZSBieXRlCj4gTWVyZ2luZy9jb21iaW5pbmcgZXRjIGFzIGRlZmluZWQgaW4gdGhl IFBDSSBzcGVjLCBhbmQgcGVyIGEgcmF0aGVyIHBvb3JseQo+IHdvcmRlZCBJbXBsZW1lbnRhdGlv biBub3RlIGluIHRoZSBzcGVjIHNheXMgdGhhdCBubyBrbm93biBQQ0llIEhvc3QgQnJpZGRnZXMv Um9vdCAKPiBwb3J0cyBkbyBpdCBlaXRoZXIuIAo+IFNvIGZvciBQQ0llIHdlIGRvbid0IHRoaW5r IGJlbGlldmUgdGhlcmUgaXMgYW55IGJ5dGUgbWVyZ2luZyB0aGF0IGhhcHBlbnMgaW4gdGhlIFBD SWUKPiBmYWJyaWMgc28gaXQncyByZWFsbHkgYSBtYXR0ZXIgb2Ygd2hhdCBoYXBwZW5zIGluIHRo ZSBDUFUgY29yZSBhbmQgaW50ZXJjb25uZWN0Cj4gYmVmb3JlIGl0IGdldHMgdG8gdGhlIFBDSWUg aGllcmFyY2h5LgoKWWVzLCBidXQgbWVyZ2VkIHdyaXRlcywgbm8gbWF0dGVyIHdoZXJlIHRoZXkg aGFwcGVuLCBhcmUgc3RpbGwgdGhlIG9ubHkKdHlwZSBvZiB3cml0ZSBjb21iaW5pbmcgdGhhdCBh IHByZWZldGNoYWJsZSBCQVIgb24gYW4gZW5kcG9pbnQgaXMKcmVxdWlyZWQgdG8gc3VwcG9ydC4K Cj4gU3RlcHBpbmcgYmFjayBmcm9tIHRoaXMgcGF0Y2hzZXQsIGRvIHlvdSBhZ3JlZSB0aGF0IGl0 IGlzIGRlc2lyYWJsZSB0byBzdXBwb3J0Cj4gV3JpdGUgY29tYmluaW5nIGFzIHVuZGVyc3Rvb2Qg YnkgaW9yZW1hcF93YyB0byB3b3JrIGluIGFsbCBJU0EgZ3Vlc3RzIGluY2x1ZGluZwo+IEFSTXY4 PwoKWWVzLCBhIHVzZXJzcGFjZSB2ZmlvIGRyaXZlciBzaG91bGQgYmUgYWJsZSB0byB0YWtlIGFk dmFudGFnZSBvZiB0aGUKaGFyZHdhcmUgY2FwYWJpbGl0aWVzLiAgSSB0aGluayB3aGVyZSB3ZSBk aXNhZ3JlZSBpcyB3aGV0aGVyIGl0J3MKdW5pdmVyc2FsbHkgc2FmZSB0byBhc3N1bWUgd3JpdGUg Y29tYmluaW5nIGJhc2VkIG9uIHRoZSBQQ0kKcHJlZmV0Y2hhYmxlIGNhcGFiaWxpdHkgb2YgYSBC QVIuICBJZiB0aGF0J3Mgc29tZXRoaW5nIHRoYXQgY2FuIGJlCmFzc3VtZWQgdW5pdmVyc2FsbHkg Zm9yIEFSTXY4IGJhc2VkIG9uIHRoZSBhcmNoaXRlY3R1cmUgc3BlY2lmaWNhdGlvbgpjb21wYXRp YmlsaXR5IHdpdGggdGhlIFBDSSBkZWZpbml0aW9uIG9mIGEgcHJlZmV0Y2hhYmxlIEJBUiwgdGhl biBJCndvdWxkIGV4cGVjdCBhIGhlbHBlciBzb21ld2hlcmUgaW4gYXJjaCBjb2RlIHRoYXQgcmV0 dXJucyB0aGUgcmlnaHQKcGFnZSBwcm90ZWN0aW9uIGZsYWdzLCBzbyB0aGF0IGFyY2ggbWFpbnRh aW5lcnMgZG9uJ3QgbmVlZCB0byBzY291cgpkZXZpY2UgZHJpdmVycyBmb3IgYXJjaGl0ZWN0dXJl IGhhY2tzLiAgT3RoZXJ3aXNlLCBpdCBuZWVkcyB0byBiZQpleHBvc2VkIHRocm91Z2ggdGhlIHZm aW8gdUFQSSB0byBhbGxvdyB0aGUgdXNlcnNwYWNlIGRldmljZSBkcml2ZXIKaXRzZWxmIHRvIHNl bGVjdCB0aGVzZSBzZW1hbnRpY3MuCgo+IFlvdSBub3RlIHRoYXQgeDg2IHZpcnR1YWxpemF0aW9u IGRvZXNuJ3QgaGF2ZSB0aGlzIGlzc3VlLCBidXQgS1ZNLUFSTSBkb2VzCj4gYmVjYXVzZSBLVk0g bWFwcyBhbGwgZGV2aWNlIEJBUnMgYXMgRGV2aWNlIE1lbW9yeSB0eXBlIG5HblJFIHdoaWNoIAo+ IGRvZXNuJ3QgYWxsb3cgaW9yZW1hcF93YyBmcm9tIHdpdGhpbiB0aGUgZ3Vlc3QgdG8gZ2V0IHRo ZSBhY3R1YWwgc2VtYW50aWNzIGRlc2lyZWQuCj4gCj4gTWFyYyBhbmQgb3RoZXJzIGhhdmUgc3Vn Z2VzdGVkIHRoYXQgdXNlcnNwYWNlIHNob3VsZCBwcm92aWRlIHRoZSBoaW50cy4gQnV0IHRoZQo+ IHF1ZXN0aW9uIGlzIGhvdyB3b3VsZCBxZW11IHZmaW8gZG8gdGhpcyBlaXRoZXI/IFdlIHdvdWxk IGJlIHN0dWNrIGluIHRoZSBzYW1lCj4gYXJndW1lbnRzIGFzIGhlcmUsIGFzIHRvIHdoYXQgaXMg dGhlIGNvcnJlY3Qgd2F5IHRvIGRldGVybWluZSB0aGUgZGVzaXJlZCBhdHRyaWJ1dGVzCj4gZm9y IGEgZ2l2ZW4gQkFSIHN1Y2ggdGhhdCBldmVudHVhbGx5IHdoZW4gYSBkcml2ZXIgaW4gdGhlIGd1 ZXN0IGFza3MgZm9yCj4gaW9yZW1hcF93YyBpdCBhY3R1YWxseSBoYXMgYSBjaGFuY2Ugb2Ygd29y a2luZyBpbiB0aGUgZ3Vlc3QsIGluIGFsbCBJU0FzLiAKPiBEbyB5b3UgaGF2ZSBhbnkgc3VnZ2Vz dGlvbnMgb24gaG93IHRvIG1ha2UgcHJvZ3Jlc3MgaGVyZT8KCldlIGRvIG5lZWQgc29tZSB3YXkg Zm9yIHVzZXJzcGFjZSBkcml2ZXJzIHRvIGFsc28gbWFrZSB1c2Ugb2YgV0MKc2VtYW50aWNzLCB0 aGVyZSB3ZXJlIHNvbWUgZGlzY3Vzc2lvbnMgaW4gdGhlIHBhc3QsIEkgdGhpbmsgb3RoZXJzIGhh dmUKcmVmZXJlbmNlZCB0aGVtIGFzIHdlbGwsIGJ1dCBub3RoaW5nIGhhcyBiZWVuIHByb3Bvc2Vk IGZvciBhIHZmaW8gQVBJLgoKSWYgd2UgaGFkIHRoYXQgQVBJLCBRRU1VIGRlY2lkaW5nIHRvIHVu aXZlcnNhbGx5IGVuYWJsZSBXQyBmb3IgYWxsCnZmaW8gcHJlZmV0Y2hhYmxlIEJBUnMgc2VlbXMg b25seSBtYXJnaW5hbGx5IGJldHRlciB0aGFuIHRoaXMgYXBwcm9hY2guClVsdGltYXRlbHkgdGhl IG1hcHBpbmcgc2hvdWxkIGJlIGJhc2VkIG9uIHRoZSBndWVzdCBkcml2ZXIgc2VtYW50aWNzLAph bmQgaWYgeW91IGRvbid0IGhhdmUgYW55IHZpc2liaWxpdHkgdG8gdGhhdCBvbiBLVk0vYXJtIGxp a2Ugd2UgaGF2ZSBvbgpLVk0veDg2LCB0aGVuIGl0IHNlZW1zIGxpa2UgdGhlcmUncyBub3RoaW5n IHRvIHRyaWdnZXIgYSB2ZmlvIEFQSSBoZXJlCmFueXdheS4KCklmIHRoYXQncyB0aGUgY2FzZSwg SSdkIHByb2JhYmx5IGdvIGJhY2sgdG8gbGV0dGluZyB0aGUgYXJjaC9hcm02NCBmb2xrcwpkZWNs YXJlIHRoYXQgV0MgaXMgY29tcGF0aWJsZSB3aXRoIHRoZSBkZWZpbml0aW9uIG9mIFBDSSBwcmVm ZXRjaGFibGUKYW5kIGV4cG9ydCBzb21lIHNvcnQgb2YgcGdwcm90X3BjaV9wcmVmZXRjaGFibGUo KSBoZWxwZXIgd2hlcmUgdGhlCmRlZmF1bHQgd291bGQgYmUgdG8gI2RlZmluZSBpdCBhcyBwZ3By b2Nfbm9uY2FjaGVkKCkgI2lmbmRlZiBieSB0aGUKYXJjaC4KCj4gQSBkZXZpY2Ugc3BlY2lmaWMg bGlzdCBvZiB3aGljaCBCQVJzIGFyZSBPSyB0byBhbGxvdyBpb3JlbWFwX3djIGZvciBzZWVtcyB0 ZXJyaWJsZQo+IGFuZCBJJ20gbm90IHN1cmUgaWYgYSBjb21tYW5kbGluZSBxZW11IG9wdGlvbiBp cyBhbnkgYmV0dGVyLiBJcyB0aGUgdXNlciBvZiBkZXZpY2UgCj4gYXNzaWdubWVudC9zeXNhZG1p biBzdXBwb3NlZCB0byBrbm93IHdoaWNoIEJBUiBvZiB3aGljaCBkZXZpY2UgaXMgT0sgdG8gYWxs b3cgCj4gaW9yZW1hcF93YyBmb3I/CgpObywgYSBkZXZpY2Ugc3BlY2lmaWMgdXNlcnNwYWNlIGRy aXZlciBzaG91bGQga25vdyBzdWNoIGRldmljZQpzZW1hbnRpY3MsIGJ1dCBRRU1VIGlzIG5vdCBz dWNoIGEgZHJpdmVyLiAgQnVyZGVuaW5nIHRoZSBoeXBlcnZpc29yCnVzZXIvYWRtaW4gaXMgbm90 IGEgZ29vZCBzb2x1dGlvbiBlaXRoZXIuICBJJ2QgbGVhbiBvbiBLVk0vYXJtNjQgZm9sa3MKdG8g a25vdyBob3cgdGhlIGd1ZXN0IGRyaXZlciBzZW1hbnRpY3MgY2FuIGJlIGV4cG9zZWQgdG8gdGhl Cmh5cGVydmlzb3IuICBUaGFua3MsCgpBbGV4CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fXwprdm1hcm0gbWFpbGluZyBsaXN0Cmt2bWFybUBsaXN0cy5jcy5j b2x1bWJpYS5lZHUKaHR0cHM6Ly9saXN0cy5jcy5jb2x1bWJpYS5lZHUvbWFpbG1hbi9saXN0aW5m by9rdm1hcm0K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12165C433B4 for ; Tue, 4 May 2021 18:05:47 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73F6D613CF for ; Tue, 4 May 2021 18:05:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73F6D613CF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=HWR2Z8Pvgv4UfzsG6PbOtxJJVhLF3Z/9HSdhpv8dkNk=; b=I3ERONA9L1+c1/15bHmf3p2IY v+m8S7Tl/REvocfX6viurYQSMyxzADXIpp8EUvCRGyPe0MtARNCbgKzjol+SC2AG0YaUWSsjyw5Va EhtkwLLRt74BFI3ZoEpgIHXoH7noPYvJnc95z5RkcXzZ/eOvQZokP+dLvAHaE1joBzGKCzg9Cdf7F wbQ0O/ugLhK6R9fmjGKgBRrtfs9EfLdCmMk+Yz2WRn3iDtRVi4Q9ZXcOxtc5SuPEUO5RF5f1uInzO FZo66RdGfhQ12W1vzy2qVN9EKqVPt81aGfJVmN5lfNr5StimisI/gnGjFrQSIVGE3M+ta0vEPrF7z wIwWM61oQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldzOw-00GjuI-BW; Tue, 04 May 2021 18:04:06 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldzOt-00Gjtw-JJ for linux-arm-kernel@desiato.infradead.org; Tue, 04 May 2021 18:04:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To: From:Date:Sender:Reply-To:Content-ID:Content-Description; bh=crIjxSKc9RicXz5uU0iWJNjyfJ75605vwRv4rA88bAc=; b=NO/F1mHgdbq/5SASsADGFFbmwv qyuZvXBAQ9U6rqtDiqCI9LSHRkrJUG6X/Y3oBbOxYH0kLJFq6F94BWzO0MQjOl8PhPN7P/ZPuyvH4 9lwa3+mp5tePz9Vju8do+DIaVM5FWpE4c7pCZzYIo7/0JW0D5KfDSxSr7UPMBQNWZlBG4OJVgieQd TGd65+JoSNw21sL+xlScTYkWbZ1ZaEnLM9jeZ7hYPBJnMRR3NMpkhRBe6Hprehdvy+v0Ji/s2i55P //NpFebegNEf8oNuSSX8Qc4+YFbZPs8uN6UTlQLY1CF0gJ9wwwSNwj+tF4jQ5ymBOojClUH4MwnBY hT0o3NzA==; Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldzOq-004A6F-Mb for linux-arm-kernel@lists.infradead.org; Tue, 04 May 2021 18:04:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620151435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=crIjxSKc9RicXz5uU0iWJNjyfJ75605vwRv4rA88bAc=; b=VJ02Zb1U/jDECeflEibDMOpJor31d3nN2LtJsuQne3DkhWfrwyrsDQaQqJCdo8SRPLisA4 twY49ScGb1KBTlr28wjSciN3gczv+km9iRtt/svRKBKSlrAfiYANssnB+9RANZMnYlcrxa /Ypyyk+Ppnb5Ulhj8KuOxOYwDQUWIBc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-538-UeGCIxtyO5OSuVhwk3SZ8A-1; Tue, 04 May 2021 14:03:51 -0400 X-MC-Unique: UeGCIxtyO5OSuVhwk3SZ8A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BE4AA107ACED; Tue, 4 May 2021 18:03:49 +0000 (UTC) Received: from redhat.com (ovpn-113-225.phx2.redhat.com [10.3.113.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F9921002388; Tue, 4 May 2021 18:03:49 +0000 (UTC) Date: Tue, 4 May 2021 12:03:48 -0600 From: Alex Williamson To: Vikram Sethi Cc: Mark Kettenis , Marc Zyngier , Shanker Donthineni , "will@kernel.org" , "catalin.marinas@arm.com" , "christoffer.dall@arm.com" , "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Jason Sequeira Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA Message-ID: <20210504120348.2eec075b@redhat.com> In-Reply-To: References: <20210429162906.32742-1-sdonthineni@nvidia.com> <20210429162906.32742-2-sdonthineni@nvidia.com> <20210429122840.4f98f78e@redhat.com> <470360a7-0242-9ae5-816f-13608f957bf6@nvidia.com> <20210429134659.321a5c3c@redhat.com> <87czucngdc.wl-maz@kernel.org> <1edb2c4e-23f0-5730-245b-fc6d289951e1@nvidia.com> <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210504_110400_840302_DC80E919 X-CRM114-Status: GOOD ( 49.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gTW9uLCAzIE1heSAyMDIxIDIyOjAzOjU5ICswMDAwClZpa3JhbSBTZXRoaSA8dnNldGhpQG52 aWRpYS5jb20+IHdyb3RlOgoKPiBIaSBBbGV4LAo+ID4gRnJvbTogQWxleCBXaWxsaWFtc29uIDxh bGV4LndpbGxpYW1zb25AcmVkaGF0LmNvbT4KPiA+IE9uIE1vbiwgMyBNYXkgMjAyMSAxMzo1OTo0 MyArMDAwMAo+ID4gVmlrcmFtIFNldGhpIDx2c2V0aGlAbnZpZGlhLmNvbT4gd3JvdGU6ICAKPiA+ ID4gPiBGcm9tOiBNYXJrIEtldHRlbmlzIDxtYXJrLmtldHRlbmlzQHhzNGFsbC5ubD4gIAo+ID4g PiA+ID4gRnJvbTogTWFyYyBaeW5naWVyIDxtYXpAa2VybmVsLm9yZz4gIAo+ID4gPgo+ID4gPiBz bmlwICAKPiA+ID4gPiA+IElmLCBieSBlbnVtZXJhdGluZyB0aGUgcHJvcGVydGllcyBvZiBQcmVm ZXRjaGFibGUsIHlvdSBjYW4gc2hvdwo+ID4gPiA+ID4gdGhhdCB0aGV5IGFyZSBhIHN0cmljdCBz dXBlcnNldCBvZiBOb3JtYWxfTkMsIEknbSBvbiBib2FyZC4gSQo+ID4gPiA+ID4gaGF2ZW4ndCBz ZWVuIHN1Y2ggYW4gZW51bWVyYXRpb24gc28gZmFyLgo+ID4gPiA+ID4gIAo+ID4gPiBzbmlwICAK PiA+ID4gPiA+IFJpZ2h0LCBzbyB3ZSBoYXZlIG1hZGUgYSBzbWFsbCBzdGVwIGluIHRoZSBkaXJl Y3Rpb24gb2YgbWFwcGluZwo+ID4gPiA+ID4gInByZWZldGNoYWJsZSIgb250byAiTm9ybWFsX05D IiwgdGhhbmtzIGZvciB0aGF0LiBXaGF0IGFib3V0IGFsbAo+ID4gPiA+ID4gdGhlIG90aGVyIHBy b3BlcnRpZXMgKHVuYWxpZ25lZCBhY2Nlc3Nlcywgb3JkZXJpbmcsIGdhdGhlcmluZyk/ICAKPiA+ ID4gPiAgCj4gPiA+IFJlZ2FyZGluZyBnYXRoZXJpbmcvd3JpdGUgY29tYmluaW5nLCB0aGF0IGlz IGFsc28gYWxsb3dlZCB0bwo+ID4gPiBwcmVmZXRjaGFibGUgcGVyIFBDSSBzcGVjICAKPiA+IAo+ ID4gQXMgb3RoZXJzIGhhdmUgc3RhdGVkLCBnYXRoZXIvd3JpdGUgY29tYmluaW5nIGl0c2VsZiBp cyBub3Qgd2VsbCBkZWZpbmVkLgo+ID4gICAKPiA+ID4gRnJvbSAxLjMuMi4yIG9mIDUvMCBiYXNl IHNwZWM6Cj4gPiA+IEEgUENJIEV4cHJlc3MgRW5kcG9pbnQgcmVxdWVzdGluZyBtZW1vcnkgcmVz b3VyY2VzIHRocm91Z2ggYSBCQVIgbXVzdAo+ID4gPiBzZXQgdGhlIEJBUidzIFByZWZldGNoYWJs ZSBiaXQgdW5sZXNzIHRoZSByYW5nZSBjb250YWlucyBsb2NhdGlvbnMKPiA+ID4gd2l0aCByZWFk IHNpZGUtZWZmZWN0cyBvciBsb2NhdGlvbnMgaW4gd2hpY2ggdGhlIEZ1bmN0aW9uIGRvZXMgbm90 IHRvbGVyYXRlICAKPiA+IHdyaXRlIG1lcmdpbmcuCj4gPiAKPiA+ICJ3cml0ZSBtZXJnaW5nIiAg VGhpcyBpcyBhIHZlcnkgc3BlY2lmaWMgdGhpbmcsIHBlciBQQ0kgMy4wLCAzLjIuNjoKPiA+IAo+ ID4gICBCeXRlIE1lcmdpbmcg4oCTIG9jY3VycyB3aGVuIGEgc2VxdWVuY2Ugb2YgaW5kaXZpZHVh bCBtZW1vcnkgd3JpdGVzCj4gPiAgIChieXRlcyBvciB3b3JkcykgYXJlIG1lcmdlZCBpbnRvIGEg c2luZ2xlIERXT1JELgo+ID4gCj4gPiBUaGUgc2VtYW50aWNzIHN1Z2dlc3QgcXVhZHdvcmQgc3Vw cG9ydCBpbiBhZGRpdGlvbiB0byBkd29yZCwgYnV0IGRvbid0Cj4gPiByZXF1aXJlIGl0LiAgV3Jp dGVzIHRvIGJ5dGVzIHdpdGhpbiBhIGR3b3JkIGNhbiBiZSBtZXJnZWQsIGJ1dCBkdXBsaWNhdGUK PiA+IHdyaXRlcyBjYW5ub3QuCj4gPiAKPiA+IEl0IHNlZW1zIGxpa2UgYW4gZXh0cmVtZWx5IGxp YmVyYWwgYXBwbGljYXRpb24gdG8gc3VnZ2VzdCB0aGF0IHRoaXMgb25lIHdyaXRlCj4gPiBzZW1h bnRpYyBlbmNvbXBhc3NlcyBmdWxsIHdyaXRlIGNvbWJpbmluZyBzZW1hbnRpY3MsIHdoaWNoIGl0 c2VsZiBpcyBub3QKPiA+IGNsZWFybHkgZGVmaW5lZC4KPiA+ICAKPiBUYWxraW5nIHRvIG91ciBQ Q0llIFNJRyByZXByZXNlbnRhdGl2ZSwgUENJZSBzd2l0Y2hlcyBhcmUgbm90IGFsbG93ZWQgZG8g YW55IG9mIHRoZSBieXRlCj4gTWVyZ2luZy9jb21iaW5pbmcgZXRjIGFzIGRlZmluZWQgaW4gdGhl IFBDSSBzcGVjLCBhbmQgcGVyIGEgcmF0aGVyIHBvb3JseQo+IHdvcmRlZCBJbXBsZW1lbnRhdGlv biBub3RlIGluIHRoZSBzcGVjIHNheXMgdGhhdCBubyBrbm93biBQQ0llIEhvc3QgQnJpZGRnZXMv Um9vdCAKPiBwb3J0cyBkbyBpdCBlaXRoZXIuIAo+IFNvIGZvciBQQ0llIHdlIGRvbid0IHRoaW5r IGJlbGlldmUgdGhlcmUgaXMgYW55IGJ5dGUgbWVyZ2luZyB0aGF0IGhhcHBlbnMgaW4gdGhlIFBD SWUKPiBmYWJyaWMgc28gaXQncyByZWFsbHkgYSBtYXR0ZXIgb2Ygd2hhdCBoYXBwZW5zIGluIHRo ZSBDUFUgY29yZSBhbmQgaW50ZXJjb25uZWN0Cj4gYmVmb3JlIGl0IGdldHMgdG8gdGhlIFBDSWUg aGllcmFyY2h5LgoKWWVzLCBidXQgbWVyZ2VkIHdyaXRlcywgbm8gbWF0dGVyIHdoZXJlIHRoZXkg aGFwcGVuLCBhcmUgc3RpbGwgdGhlIG9ubHkKdHlwZSBvZiB3cml0ZSBjb21iaW5pbmcgdGhhdCBh IHByZWZldGNoYWJsZSBCQVIgb24gYW4gZW5kcG9pbnQgaXMKcmVxdWlyZWQgdG8gc3VwcG9ydC4K Cj4gU3RlcHBpbmcgYmFjayBmcm9tIHRoaXMgcGF0Y2hzZXQsIGRvIHlvdSBhZ3JlZSB0aGF0IGl0 IGlzIGRlc2lyYWJsZSB0byBzdXBwb3J0Cj4gV3JpdGUgY29tYmluaW5nIGFzIHVuZGVyc3Rvb2Qg YnkgaW9yZW1hcF93YyB0byB3b3JrIGluIGFsbCBJU0EgZ3Vlc3RzIGluY2x1ZGluZwo+IEFSTXY4 PwoKWWVzLCBhIHVzZXJzcGFjZSB2ZmlvIGRyaXZlciBzaG91bGQgYmUgYWJsZSB0byB0YWtlIGFk dmFudGFnZSBvZiB0aGUKaGFyZHdhcmUgY2FwYWJpbGl0aWVzLiAgSSB0aGluayB3aGVyZSB3ZSBk aXNhZ3JlZSBpcyB3aGV0aGVyIGl0J3MKdW5pdmVyc2FsbHkgc2FmZSB0byBhc3N1bWUgd3JpdGUg Y29tYmluaW5nIGJhc2VkIG9uIHRoZSBQQ0kKcHJlZmV0Y2hhYmxlIGNhcGFiaWxpdHkgb2YgYSBC QVIuICBJZiB0aGF0J3Mgc29tZXRoaW5nIHRoYXQgY2FuIGJlCmFzc3VtZWQgdW5pdmVyc2FsbHkg Zm9yIEFSTXY4IGJhc2VkIG9uIHRoZSBhcmNoaXRlY3R1cmUgc3BlY2lmaWNhdGlvbgpjb21wYXRp YmlsaXR5IHdpdGggdGhlIFBDSSBkZWZpbml0aW9uIG9mIGEgcHJlZmV0Y2hhYmxlIEJBUiwgdGhl biBJCndvdWxkIGV4cGVjdCBhIGhlbHBlciBzb21ld2hlcmUgaW4gYXJjaCBjb2RlIHRoYXQgcmV0 dXJucyB0aGUgcmlnaHQKcGFnZSBwcm90ZWN0aW9uIGZsYWdzLCBzbyB0aGF0IGFyY2ggbWFpbnRh aW5lcnMgZG9uJ3QgbmVlZCB0byBzY291cgpkZXZpY2UgZHJpdmVycyBmb3IgYXJjaGl0ZWN0dXJl IGhhY2tzLiAgT3RoZXJ3aXNlLCBpdCBuZWVkcyB0byBiZQpleHBvc2VkIHRocm91Z2ggdGhlIHZm aW8gdUFQSSB0byBhbGxvdyB0aGUgdXNlcnNwYWNlIGRldmljZSBkcml2ZXIKaXRzZWxmIHRvIHNl bGVjdCB0aGVzZSBzZW1hbnRpY3MuCgo+IFlvdSBub3RlIHRoYXQgeDg2IHZpcnR1YWxpemF0aW9u IGRvZXNuJ3QgaGF2ZSB0aGlzIGlzc3VlLCBidXQgS1ZNLUFSTSBkb2VzCj4gYmVjYXVzZSBLVk0g bWFwcyBhbGwgZGV2aWNlIEJBUnMgYXMgRGV2aWNlIE1lbW9yeSB0eXBlIG5HblJFIHdoaWNoIAo+ IGRvZXNuJ3QgYWxsb3cgaW9yZW1hcF93YyBmcm9tIHdpdGhpbiB0aGUgZ3Vlc3QgdG8gZ2V0IHRo ZSBhY3R1YWwgc2VtYW50aWNzIGRlc2lyZWQuCj4gCj4gTWFyYyBhbmQgb3RoZXJzIGhhdmUgc3Vn Z2VzdGVkIHRoYXQgdXNlcnNwYWNlIHNob3VsZCBwcm92aWRlIHRoZSBoaW50cy4gQnV0IHRoZQo+ IHF1ZXN0aW9uIGlzIGhvdyB3b3VsZCBxZW11IHZmaW8gZG8gdGhpcyBlaXRoZXI/IFdlIHdvdWxk IGJlIHN0dWNrIGluIHRoZSBzYW1lCj4gYXJndW1lbnRzIGFzIGhlcmUsIGFzIHRvIHdoYXQgaXMg dGhlIGNvcnJlY3Qgd2F5IHRvIGRldGVybWluZSB0aGUgZGVzaXJlZCBhdHRyaWJ1dGVzCj4gZm9y IGEgZ2l2ZW4gQkFSIHN1Y2ggdGhhdCBldmVudHVhbGx5IHdoZW4gYSBkcml2ZXIgaW4gdGhlIGd1 ZXN0IGFza3MgZm9yCj4gaW9yZW1hcF93YyBpdCBhY3R1YWxseSBoYXMgYSBjaGFuY2Ugb2Ygd29y a2luZyBpbiB0aGUgZ3Vlc3QsIGluIGFsbCBJU0FzLiAKPiBEbyB5b3UgaGF2ZSBhbnkgc3VnZ2Vz dGlvbnMgb24gaG93IHRvIG1ha2UgcHJvZ3Jlc3MgaGVyZT8KCldlIGRvIG5lZWQgc29tZSB3YXkg Zm9yIHVzZXJzcGFjZSBkcml2ZXJzIHRvIGFsc28gbWFrZSB1c2Ugb2YgV0MKc2VtYW50aWNzLCB0 aGVyZSB3ZXJlIHNvbWUgZGlzY3Vzc2lvbnMgaW4gdGhlIHBhc3QsIEkgdGhpbmsgb3RoZXJzIGhh dmUKcmVmZXJlbmNlZCB0aGVtIGFzIHdlbGwsIGJ1dCBub3RoaW5nIGhhcyBiZWVuIHByb3Bvc2Vk IGZvciBhIHZmaW8gQVBJLgoKSWYgd2UgaGFkIHRoYXQgQVBJLCBRRU1VIGRlY2lkaW5nIHRvIHVu aXZlcnNhbGx5IGVuYWJsZSBXQyBmb3IgYWxsCnZmaW8gcHJlZmV0Y2hhYmxlIEJBUnMgc2VlbXMg b25seSBtYXJnaW5hbGx5IGJldHRlciB0aGFuIHRoaXMgYXBwcm9hY2guClVsdGltYXRlbHkgdGhl IG1hcHBpbmcgc2hvdWxkIGJlIGJhc2VkIG9uIHRoZSBndWVzdCBkcml2ZXIgc2VtYW50aWNzLAph bmQgaWYgeW91IGRvbid0IGhhdmUgYW55IHZpc2liaWxpdHkgdG8gdGhhdCBvbiBLVk0vYXJtIGxp a2Ugd2UgaGF2ZSBvbgpLVk0veDg2LCB0aGVuIGl0IHNlZW1zIGxpa2UgdGhlcmUncyBub3RoaW5n IHRvIHRyaWdnZXIgYSB2ZmlvIEFQSSBoZXJlCmFueXdheS4KCklmIHRoYXQncyB0aGUgY2FzZSwg SSdkIHByb2JhYmx5IGdvIGJhY2sgdG8gbGV0dGluZyB0aGUgYXJjaC9hcm02NCBmb2xrcwpkZWNs YXJlIHRoYXQgV0MgaXMgY29tcGF0aWJsZSB3aXRoIHRoZSBkZWZpbml0aW9uIG9mIFBDSSBwcmVm ZXRjaGFibGUKYW5kIGV4cG9ydCBzb21lIHNvcnQgb2YgcGdwcm90X3BjaV9wcmVmZXRjaGFibGUo KSBoZWxwZXIgd2hlcmUgdGhlCmRlZmF1bHQgd291bGQgYmUgdG8gI2RlZmluZSBpdCBhcyBwZ3By b2Nfbm9uY2FjaGVkKCkgI2lmbmRlZiBieSB0aGUKYXJjaC4KCj4gQSBkZXZpY2Ugc3BlY2lmaWMg bGlzdCBvZiB3aGljaCBCQVJzIGFyZSBPSyB0byBhbGxvdyBpb3JlbWFwX3djIGZvciBzZWVtcyB0 ZXJyaWJsZQo+IGFuZCBJJ20gbm90IHN1cmUgaWYgYSBjb21tYW5kbGluZSBxZW11IG9wdGlvbiBp cyBhbnkgYmV0dGVyLiBJcyB0aGUgdXNlciBvZiBkZXZpY2UgCj4gYXNzaWdubWVudC9zeXNhZG1p biBzdXBwb3NlZCB0byBrbm93IHdoaWNoIEJBUiBvZiB3aGljaCBkZXZpY2UgaXMgT0sgdG8gYWxs b3cgCj4gaW9yZW1hcF93YyBmb3I/CgpObywgYSBkZXZpY2Ugc3BlY2lmaWMgdXNlcnNwYWNlIGRy aXZlciBzaG91bGQga25vdyBzdWNoIGRldmljZQpzZW1hbnRpY3MsIGJ1dCBRRU1VIGlzIG5vdCBz dWNoIGEgZHJpdmVyLiAgQnVyZGVuaW5nIHRoZSBoeXBlcnZpc29yCnVzZXIvYWRtaW4gaXMgbm90 IGEgZ29vZCBzb2x1dGlvbiBlaXRoZXIuICBJJ2QgbGVhbiBvbiBLVk0vYXJtNjQgZm9sa3MKdG8g a25vdyBob3cgdGhlIGd1ZXN0IGRyaXZlciBzZW1hbnRpY3MgY2FuIGJlIGV4cG9zZWQgdG8gdGhl Cmh5cGVydmlzb3IuICBUaGFua3MsCgpBbGV4CgoKX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgt YXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3Jn L21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37C52C433ED for ; Tue, 4 May 2021 18:03:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0ECF1610EA for ; Tue, 4 May 2021 18:03:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232047AbhEDSEu (ORCPT ); Tue, 4 May 2021 14:04:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31978 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231635AbhEDSEu (ORCPT ); Tue, 4 May 2021 14:04:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620151434; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=crIjxSKc9RicXz5uU0iWJNjyfJ75605vwRv4rA88bAc=; b=gpHXQycL4j1chUFogkUq3N3XYM51I0LG5ZdwWX48Dko7I3NabRp6/taOl6StPF0HX8x6C5 XSH7sSZm7qvFLMfppTaOkjF2+mlpSe6pyKJrVzhb5u8HomiNge0XTT1ZE5B7Dir4tc6MKo O8Gr3kxW2y6wLmNaY0/qiEoM7lQJuWY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-538-UeGCIxtyO5OSuVhwk3SZ8A-1; Tue, 04 May 2021 14:03:51 -0400 X-MC-Unique: UeGCIxtyO5OSuVhwk3SZ8A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BE4AA107ACED; Tue, 4 May 2021 18:03:49 +0000 (UTC) Received: from redhat.com (ovpn-113-225.phx2.redhat.com [10.3.113.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F9921002388; Tue, 4 May 2021 18:03:49 +0000 (UTC) Date: Tue, 4 May 2021 12:03:48 -0600 From: Alex Williamson To: Vikram Sethi Cc: Mark Kettenis , Marc Zyngier , Shanker Donthineni , "will@kernel.org" , "catalin.marinas@arm.com" , "christoffer.dall@arm.com" , "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Jason Sequeira Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA Message-ID: <20210504120348.2eec075b@redhat.com> In-Reply-To: References: <20210429162906.32742-1-sdonthineni@nvidia.com> <20210429162906.32742-2-sdonthineni@nvidia.com> <20210429122840.4f98f78e@redhat.com> <470360a7-0242-9ae5-816f-13608f957bf6@nvidia.com> <20210429134659.321a5c3c@redhat.com> <87czucngdc.wl-maz@kernel.org> <1edb2c4e-23f0-5730-245b-fc6d289951e1@nvidia.com> <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Mon, 3 May 2021 22:03:59 +0000 Vikram Sethi wrote: > Hi Alex, > > From: Alex Williamson > > On Mon, 3 May 2021 13:59:43 +0000 > > Vikram Sethi wrote: =20 > > > > From: Mark Kettenis =20 > > > > > From: Marc Zyngier =20 > > > > > > snip =20 > > > > > If, by enumerating the properties of Prefetchable, you can show > > > > > that they are a strict superset of Normal_NC, I'm on board. I > > > > > haven't seen such an enumeration so far. > > > > > =20 > > > snip =20 > > > > > Right, so we have made a small step in the direction of mapping > > > > > "prefetchable" onto "Normal_NC", thanks for that. What about all > > > > > the other properties (unaligned accesses, ordering, gathering)? = =20 > > > > =20 > > > Regarding gathering/write combining, that is also allowed to > > > prefetchable per PCI spec =20 > >=20 > > As others have stated, gather/write combining itself is not well define= d. > > =20 > > > From 1.3.2.2 of 5/0 base spec: > > > A PCI Express Endpoint requesting memory resources through a BAR must > > > set the BAR's Prefetchable bit unless the range contains locations > > > with read side-effects or locations in which the Function does not to= lerate =20 > > write merging. > >=20 > > "write merging" This is a very specific thing, per PCI 3.0, 3.2.6: > >=20 > > Byte Merging =E2=80=93 occurs when a sequence of individual memory wr= ites > > (bytes or words) are merged into a single DWORD. > >=20 > > The semantics suggest quadword support in addition to dword, but don't > > require it. Writes to bytes within a dword can be merged, but duplicate > > writes cannot. > >=20 > > It seems like an extremely liberal application to suggest that this one= write > > semantic encompasses full write combining semantics, which itself is not > > clearly defined. > > =20 > Talking to our PCIe SIG representative, PCIe switches are not allowed do = any of the byte > Merging/combining etc as defined in the PCI spec, and per a rather poorly > worded Implementation note in the spec says that no known PCIe Host Bridd= ges/Root=20 > ports do it either.=20 > So for PCIe we don't think believe there is any byte merging that happens= in the PCIe > fabric so it's really a matter of what happens in the CPU core and interc= onnect > before it gets to the PCIe hierarchy. Yes, but merged writes, no matter where they happen, are still the only type of write combining that a prefetchable BAR on an endpoint is required to support. > Stepping back from this patchset, do you agree that it is desirable to su= pport > Write combining as understood by ioremap_wc to work in all ISA guests inc= luding > ARMv8? Yes, a userspace vfio driver should be able to take advantage of the hardware capabilities. I think where we disagree is whether it's universally safe to assume write combining based on the PCI prefetchable capability of a BAR. If that's something that can be assumed universally for ARMv8 based on the architecture specification compatibility with the PCI definition of a prefetchable BAR, then I would expect a helper somewhere in arch code that returns the right page protection flags, so that arch maintainers don't need to scour device drivers for architecture hacks. Otherwise, it needs to be exposed through the vfio uAPI to allow the userspace device driver itself to select these semantics. > You note that x86 virtualization doesn't have this issue, but KVM-ARM does > because KVM maps all device BARs as Device Memory type nGnRE which=20 > doesn't allow ioremap_wc from within the guest to get the actual semantic= s desired. >=20 > Marc and others have suggested that userspace should provide the hints. B= ut the > question is how would qemu vfio do this either? We would be stuck in the = same > arguments as here, as to what is the correct way to determine the desired= attributes > for a given BAR such that eventually when a driver in the guest asks for > ioremap_wc it actually has a chance of working in the guest, in all ISAs.= =20 > Do you have any suggestions on how to make progress here? We do need some way for userspace drivers to also make use of WC semantics, there were some discussions in the past, I think others have referenced them as well, but nothing has been proposed for a vfio API. If we had that API, QEMU deciding to universally enable WC for all vfio prefetchable BARs seems only marginally better than this approach. Ultimately the mapping should be based on the guest driver semantics, and if you don't have any visibility to that on KVM/arm like we have on KVM/x86, then it seems like there's nothing to trigger a vfio API here anyway. If that's the case, I'd probably go back to letting the arch/arm64 folks declare that WC is compatible with the definition of PCI prefetchable and export some sort of pgprot_pci_prefetchable() helper where the default would be to #define it as pgproc_noncached() #ifndef by the arch. > A device specific list of which BARs are OK to allow ioremap_wc for seems= terrible > and I'm not sure if a commandline qemu option is any better. Is the user = of device=20 > assignment/sysadmin supposed to know which BAR of which device is OK to a= llow=20 > ioremap_wc for? No, a device specific userspace driver should know such device semantics, but QEMU is not such a driver. Burdening the hypervisor user/admin is not a good solution either. I'd lean on KVM/arm64 folks to know how the guest driver semantics can be exposed to the hypervisor. Thanks, Alex