From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E529AC47092 for ; Wed, 2 Jun 2021 09:38:12 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 5D1FE613D3 for ; Wed, 2 Jun 2021 09:38:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5D1FE613D3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E23BC40FC7; Wed, 2 Jun 2021 05:38:11 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xDpJY0OgXPIe; Wed, 2 Jun 2021 05:38:07 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 2C948409DD; Wed, 2 Jun 2021 05:38:07 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 1402E40856 for ; Wed, 2 Jun 2021 05:38:06 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rbNQitOZu165 for ; Wed, 2 Jun 2021 05:38:02 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 0379B40839 for ; Wed, 2 Jun 2021 05:38:01 -0400 (EDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0467E60FF2; Wed, 2 Jun 2021 09:38:01 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loNK2-004zX9-Vi; Wed, 02 Jun 2021 10:37:59 +0100 Date: Wed, 02 Jun 2021 10:37:58 +0100 Message-ID: <878s3s1ua1.wl-maz@kernel.org> From: Marc Zyngier To: Shanker R Donthineni Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA In-Reply-To: <273ba1c2-dfe6-7dc1-3e40-03398e82469b@nvidia.com> References: <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> <20210504083005.GA12290@willie-the-truck> <20210505180228.GA3874@arm.com> <273ba1c2-dfe6-7dc1-3e40-03398e82469b@nvidia.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: sdonthineni@nvidia.com, catalin.marinas@arm.com, will@kernel.org, vsethi@nvidia.com, alex.williamson@redhat.com, mark.kettenis@xs4all.nl, christoffer.dall@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, jsequeira@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: "kvm@vger.kernel.org" , Catalin Marinas , "linux-kernel@vger.kernel.org" , Vikram Sethi , Alex Williamson , Jason Sequeira , Will Deacon , "kvmarm@lists.cs.columbia.edu" , "linux-arm-kernel@lists.infradead.org" , Mark Kettenis X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu SGkgU2hhbmtlciwKCk9uIFNhdCwgMDggTWF5IDIwMjEgMTc6MzM6MTEgKzAxMDAsClNoYW5rZXIg UiBEb250aGluZW5pIDxzZG9udGhpbmVuaUBudmlkaWEuY29tPiB3cm90ZToKPiAKPiBIaSBNYXJj LAo+IAo+IE9uIDUvNS8yMSAxOjAyIFBNLCBDYXRhbGluIE1hcmluYXMgd3JvdGU6Cj4gPj4+IFdp bGwvQ2F0YWxpbiwgcGVyaGFwcyB5b3UgY291bGQgZXhwbGFpbiB5b3VyIHRob3VnaHQgcHJvY2Vz cyBvbiB3aHkgeW91IGNob3NlCj4gPj4+IE5vcm1hbCBOQyBmb3IgaW9yZW1hcF93YyBvbiB0aGUg YXJtdjggbGludXggcG9ydCBpbnN0ZWFkIG9mIERldmljZSBHUkUgb3Igb3RoZXIKPiA+Pj4gRGV2 aWNlIEd4eC4KPiA+PiBJIHRoaW5rIGEgY29tYmluYXRpb24gb2Y6IGNvbXBhdGliaWxpdHkgd2l0 aCAzMi1iaXQgQXJtLCB0aGUgbmVlZCB0bwo+ID4+IHN1cHBvcnQgdW5hbGlnbmVkIGFjY2Vzc2Vz IGFuZCB0aGUgcG90ZW50aWFsIGZvciBoaWdoZXIgcGVyZm9ybWFuY2UuCj4gPiBJSVJDIHRoZSBf d2Mgc3VmZml4IGFsc28gbWF0Y2hlcyB0aGUgcGdwcm90X3dyaXRlY29tYmluZSgpIHVzZWQgYnkg c29tZQo+ID4gZHJpdmVycyB0byBtYXAgYSB2aWRlbyBmcmFtZWJ1ZmZlciBpbnRvIHVzZXIgc3Bh Y2UuIEFjY2Vzc2VzIHRvIHRoZQo+ID4gZnJhbWVidWZmZXIgYXJlIG5vdCBndWFyYW50ZWVkIHRv IGJlIGFsaWduZWQgKG1lbXNldC9tZW1jcHkgZG9uJ3QgZW5zdXJlCj4gPiBhbGlnbm1lbnQgb24g YXJtNjQgYW5kIHRoZSB1c2VyIGRvZXNuJ3QgaGF2ZSBhIG1lbXNldF9pbyBvciBtZW1jcHlfdG9p bykuCj4gPgo+ID4+IEZ1cnRoZXJtb3JlLCBpb3JlbWFwKCkgYWxyZWFkeSBnaXZlcyB5b3UgYSBE ZXZpY2UgbWVtb3J5IHR5cGUsIGFuZCB3ZSdyZQo+ID4+IHRpZ2h0IG9uIE1BSVIgc3BhY2UuCj4g PiBXZSBoYXZlIE1UX0RFVklDRV9HUkUgY3VycmVudGx5IHJlc2VydmVkIHRob3VnaCBubyBpbi1r ZXJuZWwgdXNlciwgd2UKPiA+IG1pZ2h0IGFzIHdlbGwgcmVtb3ZlIGl0Lgo+IEBNYXJjLCBDb3Vs ZCB5b3UgcHJvdmlkZSB5b3VyIHRob3VnaHRzL2d1aWRhbmNlIGZvciB0aGUgbmV4dCBzdGVwPyBU aGUKPiBwcm9wb3NhbCBvZiBnZXR0aW5nIGhpbnRzIGZvciBwcmVmZXRjaGFibGUgcmVnaW9ucyBm cm9tIFZGSU8vUUVNVSBpcyBub3QKPiByZWNvbW1lbmRlZCwgVGhlIG9ubHkgb3B0aW9uIGxlZnQg aXMgdG8gaW1wbGVtZW50IEFSTTY0IGRlcGVuZGVudCBsb2dpYwo+IGluIEtWTS4KPiAKPiBPcHRp b24tMTogSSB0aGluayB3ZSBjb3VsZCB0YWtlIGFkdmFudGFnZSBvZiBzdGFnZS0xLzIgY29tYmlu aW5nIHJ1bGVzIHRvCj4gYWxsb3cgTk9STUFMX05DIG1lbW9yeS10eXBlIGZvciBkZXZpY2UgbWVt b3J5IGluIFZNLiBBbHdheXMgbWFwCj4gZGV2aWNlIG1lbW9yeSBhdCBzdGFnZS0yIGFzIE5PUk1B TC1OQyBhbmQgdHJ1c3QgVk0ncyBzdGFnZS0xIE1ULgo+IAo+IC0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQo+IFN0YWdlLTIgTVTC oMKgwqDCoCBTdGFnZS0xIE1UwqDCoMKgIFJlc3VsdGFudCBNVCAoY29tYmluaW5nLXJ1bGVzL0ZX QikKPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0KPiBOb3JtYWwtTkPCoMKgwqDCoMKgIE5vcm1hbC1XVMKgwqDCoMKgwqDCoMKg wqDCoMKgIE5vcm1hbC1OQwo+IMKgwqAgLcKgwqDCoMKgwqDCoMKgwqDCoMKgIE5vcm1hbC1XQsKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIC0KPiDCoMKgIC3CoMKgwqDCoMKgwqDCoMKgwqDCoCBO b3JtYWwtTkPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAtCj4gwqDCoCAtwqDCoMKgwqDCoMKg wqDCoMKgwqAgRGV2aWNlLTxhdHRyPsKgwqDCoMKgwqDCoCBEZXZpY2UtPGF0dHI+Cj4gLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t CgpJIHRoaW5rIHRoaXMgaXMgdW53aXNlLgoKV2lsbCByZWNlbnRseSBkZWJ1Z2dlZCBhIHByZXR0 eSBob3JyaWJsZSBzaXR1YXRpb24gd2hlbiBkb2luZyBleGFjdGx5CnRoYXQ6IHdoZW4gUzEgaXMg b2ZmIGFuZCBTMiBpcyBvbiwgdGhlIEktc2lkZSBpcyBhbGxvd2VkIHRvIGdlbmVyYXRlCnNwZWN1 bGF0aXZlIGFjY2Vzc2VzIChzZWUgQVJNdjggQVJNIEcuYSBENS4yLjkgZm9yIHRoZSBkZXRhaWxz KS4gQW5kCnllcywgaW1wbGVtZW50YXRpb25zIGRlZmluaXRlbHkgZG8gdGhhdC4gQWRkIHNpZGUt ZWZmZWN0IHJlYWRzIHRvIHRoZQptaXgsIGFuZCB5b3UncmUgaW4gZm9yIGEgdHJlYXQuCgo+IFdl J3ZlIGJlZW4gdXNpbmcgdGhpcyBvcHRpb24gaW50ZXJuYWxseSBmb3IgdGVzdGluZyBwdXJwb3Nl IGFuZAo+IHZhbGlkYXRlZCB3aXRoIE5WTUUvTWVsbGFub3gvR1BVIHBhc3MtdGhyb3VnaCBkZXZp Y2VzIG9uCj4gTWFydmVsbC1UaHVuZGV4MiBwbGF0Zm9ybS4KClNlZSBhYm92ZS4gSXQgKndpbGwq IGJyZWFrIGV2ZW50dWFsbHkuCgo+IE9wdGlvbi0yOiBHZXQgcmVzb3VyY2UgcHJvcGVydGllcyBh c3NvY2lhdGVkIHdpdGggTU1JTyB1c2luZyBsb29rdXBfcmVzb3VyY2UoKQo+IGFuZCBtYXAgYXQg c3RhZ2UtMiBhcyBOb3JtYWwtTkMgaWYgSU9SRVNPVVJDRV9QUkVGRVRDSCBpcyBzZXQgaW4gZmxh Z3MuCgpUaGF0J3MgYSBwcmV0dHkgcm91bmRhYm91dCB3YXkgb2YgZG9pbmcgZXhhY3RseSB0aGUg c2FtZSB0aGluZyB5b3UKaW5pdGlhbGx5IHByb3Bvc2VkLiBBbmQgaXQgc3VmZmVycyBmcm9tIHRo ZSBleGFjdCBzYW1lIHByb2JsZW1zLCB3aGljaAppcyB0aGF0IHlvdSBjaGFuZ2UgdGhlIHNlbWFu dGljcyBvZiB0aGUgbWFwcGluZyB3aXRob3V0IGtub3dpbmcgd2hhdAp0aGUgZ3Vlc3QncyBpbnRl bnQgaXMuCgoJTS4KCi0tIApXaXRob3V0IGRldmlhdGlvbiBmcm9tIHRoZSBub3JtLCBwcm9ncmVz cyBpcyBub3QgcG9zc2libGUuCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fCmt2bWFybSBtYWlsaW5nIGxpc3QKa3ZtYXJtQGxpc3RzLmNzLmNvbHVtYmlhLmVk dQpodHRwczovL2xpc3RzLmNzLmNvbHVtYmlhLmVkdS9tYWlsbWFuL2xpc3RpbmZvL2t2bWFybQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00E54C4708F for ; Wed, 2 Jun 2021 09:39:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C720D613BF for ; Wed, 2 Jun 2021 09:39:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C720D613BF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hkdeafGsRC8JblWO2UAGC74VzjaTkfB+JBnpUwMirCk=; b=YF8t+Xypt82yP0 xrzPILinFXgfM+7Ycfi1+Kq/dyKVSIJOSN7Cu8u4xs2wnAlruVunrQ71Z3SENrcoxfzSnYx8Vqbqh 1FFuNtUZKOBI4xS1UBH1AI0bvyG4x7uz1Meb2yOdCMVLSjj3hPjXH/ZU0J75c0ENXTuuZJXTPcNR6 fYe7pmQdKenK9ifoteOLKQz24O5w8Prusc35WA3QUii8JWNFO6GTHtuKxfoVf5g0637xWPgGi5DpR cZqnwehSYLvboRrlGBWrYXOOUpyrDCsy9yxyjbEEYAc0WCo9EtCB4bKE1JNGmpDXtpc+IT0usYARr RqqieDsWh0O8HP7cVfYA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1loNK9-0031Bc-Jo; Wed, 02 Jun 2021 09:38:05 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1loNK5-0031Ae-8L for linux-arm-kernel@lists.infradead.org; Wed, 02 Jun 2021 09:38:02 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0467E60FF2; Wed, 2 Jun 2021 09:38:01 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loNK2-004zX9-Vi; Wed, 02 Jun 2021 10:37:59 +0100 Date: Wed, 02 Jun 2021 10:37:58 +0100 Message-ID: <878s3s1ua1.wl-maz@kernel.org> From: Marc Zyngier To: Shanker R Donthineni Cc: Catalin Marinas , Will Deacon , Vikram Sethi , Alex Williamson , Mark Kettenis , "christoffer.dall@arm.com" , "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Jason Sequeira Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA In-Reply-To: <273ba1c2-dfe6-7dc1-3e40-03398e82469b@nvidia.com> References: <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> <20210504083005.GA12290@willie-the-truck> <20210505180228.GA3874@arm.com> <273ba1c2-dfe6-7dc1-3e40-03398e82469b@nvidia.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: sdonthineni@nvidia.com, catalin.marinas@arm.com, will@kernel.org, vsethi@nvidia.com, alex.williamson@redhat.com, mark.kettenis@xs4all.nl, christoffer.dall@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, jsequeira@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210602_023801_351148_C50527A3 X-CRM114-Status: GOOD ( 28.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgU2hhbmtlciwKCk9uIFNhdCwgMDggTWF5IDIwMjEgMTc6MzM6MTEgKzAxMDAsClNoYW5rZXIg UiBEb250aGluZW5pIDxzZG9udGhpbmVuaUBudmlkaWEuY29tPiB3cm90ZToKPiAKPiBIaSBNYXJj LAo+IAo+IE9uIDUvNS8yMSAxOjAyIFBNLCBDYXRhbGluIE1hcmluYXMgd3JvdGU6Cj4gPj4+IFdp bGwvQ2F0YWxpbiwgcGVyaGFwcyB5b3UgY291bGQgZXhwbGFpbiB5b3VyIHRob3VnaHQgcHJvY2Vz cyBvbiB3aHkgeW91IGNob3NlCj4gPj4+IE5vcm1hbCBOQyBmb3IgaW9yZW1hcF93YyBvbiB0aGUg YXJtdjggbGludXggcG9ydCBpbnN0ZWFkIG9mIERldmljZSBHUkUgb3Igb3RoZXIKPiA+Pj4gRGV2 aWNlIEd4eC4KPiA+PiBJIHRoaW5rIGEgY29tYmluYXRpb24gb2Y6IGNvbXBhdGliaWxpdHkgd2l0 aCAzMi1iaXQgQXJtLCB0aGUgbmVlZCB0bwo+ID4+IHN1cHBvcnQgdW5hbGlnbmVkIGFjY2Vzc2Vz IGFuZCB0aGUgcG90ZW50aWFsIGZvciBoaWdoZXIgcGVyZm9ybWFuY2UuCj4gPiBJSVJDIHRoZSBf d2Mgc3VmZml4IGFsc28gbWF0Y2hlcyB0aGUgcGdwcm90X3dyaXRlY29tYmluZSgpIHVzZWQgYnkg c29tZQo+ID4gZHJpdmVycyB0byBtYXAgYSB2aWRlbyBmcmFtZWJ1ZmZlciBpbnRvIHVzZXIgc3Bh Y2UuIEFjY2Vzc2VzIHRvIHRoZQo+ID4gZnJhbWVidWZmZXIgYXJlIG5vdCBndWFyYW50ZWVkIHRv IGJlIGFsaWduZWQgKG1lbXNldC9tZW1jcHkgZG9uJ3QgZW5zdXJlCj4gPiBhbGlnbm1lbnQgb24g YXJtNjQgYW5kIHRoZSB1c2VyIGRvZXNuJ3QgaGF2ZSBhIG1lbXNldF9pbyBvciBtZW1jcHlfdG9p bykuCj4gPgo+ID4+IEZ1cnRoZXJtb3JlLCBpb3JlbWFwKCkgYWxyZWFkeSBnaXZlcyB5b3UgYSBE ZXZpY2UgbWVtb3J5IHR5cGUsIGFuZCB3ZSdyZQo+ID4+IHRpZ2h0IG9uIE1BSVIgc3BhY2UuCj4g PiBXZSBoYXZlIE1UX0RFVklDRV9HUkUgY3VycmVudGx5IHJlc2VydmVkIHRob3VnaCBubyBpbi1r ZXJuZWwgdXNlciwgd2UKPiA+IG1pZ2h0IGFzIHdlbGwgcmVtb3ZlIGl0Lgo+IEBNYXJjLCBDb3Vs ZCB5b3UgcHJvdmlkZSB5b3VyIHRob3VnaHRzL2d1aWRhbmNlIGZvciB0aGUgbmV4dCBzdGVwPyBU aGUKPiBwcm9wb3NhbCBvZiBnZXR0aW5nIGhpbnRzIGZvciBwcmVmZXRjaGFibGUgcmVnaW9ucyBm cm9tIFZGSU8vUUVNVSBpcyBub3QKPiByZWNvbW1lbmRlZCwgVGhlIG9ubHkgb3B0aW9uIGxlZnQg aXMgdG8gaW1wbGVtZW50IEFSTTY0IGRlcGVuZGVudCBsb2dpYwo+IGluIEtWTS4KPiAKPiBPcHRp b24tMTogSSB0aGluayB3ZSBjb3VsZCB0YWtlIGFkdmFudGFnZSBvZiBzdGFnZS0xLzIgY29tYmlu aW5nIHJ1bGVzIHRvCj4gYWxsb3cgTk9STUFMX05DIG1lbW9yeS10eXBlIGZvciBkZXZpY2UgbWVt b3J5IGluIFZNLiBBbHdheXMgbWFwCj4gZGV2aWNlIG1lbW9yeSBhdCBzdGFnZS0yIGFzIE5PUk1B TC1OQyBhbmQgdHJ1c3QgVk0ncyBzdGFnZS0xIE1ULgo+IAo+IC0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQo+IFN0YWdlLTIgTVTC oMKgwqDCoCBTdGFnZS0xIE1UwqDCoMKgIFJlc3VsdGFudCBNVCAoY29tYmluaW5nLXJ1bGVzL0ZX QikKPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0KPiBOb3JtYWwtTkPCoMKgwqDCoMKgIE5vcm1hbC1XVMKgwqDCoMKgwqDCoMKg wqDCoMKgIE5vcm1hbC1OQwo+IMKgwqAgLcKgwqDCoMKgwqDCoMKgwqDCoMKgIE5vcm1hbC1XQsKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIC0KPiDCoMKgIC3CoMKgwqDCoMKgwqDCoMKgwqDCoCBO b3JtYWwtTkPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAtCj4gwqDCoCAtwqDCoMKgwqDCoMKg wqDCoMKgwqAgRGV2aWNlLTxhdHRyPsKgwqDCoMKgwqDCoCBEZXZpY2UtPGF0dHI+Cj4gLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t CgpJIHRoaW5rIHRoaXMgaXMgdW53aXNlLgoKV2lsbCByZWNlbnRseSBkZWJ1Z2dlZCBhIHByZXR0 eSBob3JyaWJsZSBzaXR1YXRpb24gd2hlbiBkb2luZyBleGFjdGx5CnRoYXQ6IHdoZW4gUzEgaXMg b2ZmIGFuZCBTMiBpcyBvbiwgdGhlIEktc2lkZSBpcyBhbGxvd2VkIHRvIGdlbmVyYXRlCnNwZWN1 bGF0aXZlIGFjY2Vzc2VzIChzZWUgQVJNdjggQVJNIEcuYSBENS4yLjkgZm9yIHRoZSBkZXRhaWxz KS4gQW5kCnllcywgaW1wbGVtZW50YXRpb25zIGRlZmluaXRlbHkgZG8gdGhhdC4gQWRkIHNpZGUt ZWZmZWN0IHJlYWRzIHRvIHRoZQptaXgsIGFuZCB5b3UncmUgaW4gZm9yIGEgdHJlYXQuCgo+IFdl J3ZlIGJlZW4gdXNpbmcgdGhpcyBvcHRpb24gaW50ZXJuYWxseSBmb3IgdGVzdGluZyBwdXJwb3Nl IGFuZAo+IHZhbGlkYXRlZCB3aXRoIE5WTUUvTWVsbGFub3gvR1BVIHBhc3MtdGhyb3VnaCBkZXZp Y2VzIG9uCj4gTWFydmVsbC1UaHVuZGV4MiBwbGF0Zm9ybS4KClNlZSBhYm92ZS4gSXQgKndpbGwq IGJyZWFrIGV2ZW50dWFsbHkuCgo+IE9wdGlvbi0yOiBHZXQgcmVzb3VyY2UgcHJvcGVydGllcyBh c3NvY2lhdGVkIHdpdGggTU1JTyB1c2luZyBsb29rdXBfcmVzb3VyY2UoKQo+IGFuZCBtYXAgYXQg c3RhZ2UtMiBhcyBOb3JtYWwtTkMgaWYgSU9SRVNPVVJDRV9QUkVGRVRDSCBpcyBzZXQgaW4gZmxh Z3MuCgpUaGF0J3MgYSBwcmV0dHkgcm91bmRhYm91dCB3YXkgb2YgZG9pbmcgZXhhY3RseSB0aGUg c2FtZSB0aGluZyB5b3UKaW5pdGlhbGx5IHByb3Bvc2VkLiBBbmQgaXQgc3VmZmVycyBmcm9tIHRo ZSBleGFjdCBzYW1lIHByb2JsZW1zLCB3aGljaAppcyB0aGF0IHlvdSBjaGFuZ2UgdGhlIHNlbWFu dGljcyBvZiB0aGUgbWFwcGluZyB3aXRob3V0IGtub3dpbmcgd2hhdAp0aGUgZ3Vlc3QncyBpbnRl bnQgaXMuCgoJTS4KCi0tIApXaXRob3V0IGRldmlhdGlvbiBmcm9tIHRoZSBub3JtLCBwcm9ncmVz cyBpcyBub3QgcG9zc2libGUuCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVs QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9s aXN0aW5mby9saW51eC1hcm0ta2VybmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1C89C47092 for ; Wed, 2 Jun 2021 09:38:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9925613BF for ; Wed, 2 Jun 2021 09:38:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231982AbhFBJjo convert rfc822-to-8bit (ORCPT ); Wed, 2 Jun 2021 05:39:44 -0400 Received: from mail.kernel.org ([198.145.29.99]:50214 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229818AbhFBJjn (ORCPT ); Wed, 2 Jun 2021 05:39:43 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0467E60FF2; Wed, 2 Jun 2021 09:38:01 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loNK2-004zX9-Vi; Wed, 02 Jun 2021 10:37:59 +0100 Date: Wed, 02 Jun 2021 10:37:58 +0100 Message-ID: <878s3s1ua1.wl-maz@kernel.org> From: Marc Zyngier To: Shanker R Donthineni Cc: Catalin Marinas , Will Deacon , Vikram Sethi , Alex Williamson , Mark Kettenis , "christoffer.dall@arm.com" , "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Jason Sequeira Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA In-Reply-To: <273ba1c2-dfe6-7dc1-3e40-03398e82469b@nvidia.com> References: <878s4zokll.wl-maz@kernel.org> <87eeeqvm1d.wl-maz@kernel.org> <87bl9sunnw.wl-maz@kernel.org> <20210503084432.75e0126d@x1.home.shazbot.org> <20210504083005.GA12290@willie-the-truck> <20210505180228.GA3874@arm.com> <273ba1c2-dfe6-7dc1-3e40-03398e82469b@nvidia.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: sdonthineni@nvidia.com, catalin.marinas@arm.com, will@kernel.org, vsethi@nvidia.com, alex.williamson@redhat.com, mark.kettenis@xs4all.nl, christoffer.dall@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, jsequeira@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi Shanker, On Sat, 08 May 2021 17:33:11 +0100, Shanker R Donthineni wrote: > > Hi Marc, > > On 5/5/21 1:02 PM, Catalin Marinas wrote: > >>> Will/Catalin, perhaps you could explain your thought process on why you chose > >>> Normal NC for ioremap_wc on the armv8 linux port instead of Device GRE or other > >>> Device Gxx. > >> I think a combination of: compatibility with 32-bit Arm, the need to > >> support unaligned accesses and the potential for higher performance. > > IIRC the _wc suffix also matches the pgprot_writecombine() used by some > > drivers to map a video framebuffer into user space. Accesses to the > > framebuffer are not guaranteed to be aligned (memset/memcpy don't ensure > > alignment on arm64 and the user doesn't have a memset_io or memcpy_toio). > > > >> Furthermore, ioremap() already gives you a Device memory type, and we're > >> tight on MAIR space. > > We have MT_DEVICE_GRE currently reserved though no in-kernel user, we > > might as well remove it. > @Marc, Could you provide your thoughts/guidance for the next step? The > proposal of getting hints for prefetchable regions from VFIO/QEMU is not > recommended, The only option left is to implement ARM64 dependent logic > in KVM. > > Option-1: I think we could take advantage of stage-1/2 combining rules to > allow NORMAL_NC memory-type for device memory in VM. Always map > device memory at stage-2 as NORMAL-NC and trust VM's stage-1 MT. > > --------------------------------------------------------------- > Stage-2 MT     Stage-1 MT    Resultant MT (combining-rules/FWB) > --------------------------------------------------------------- > Normal-NC      Normal-WT           Normal-NC >    -           Normal-WB              - >    -           Normal-NC              - >    -           Device-       Device- > --------------------------------------------------------------- I think this is unwise. Will recently debugged a pretty horrible situation when doing exactly that: when S1 is off and S2 is on, the I-side is allowed to generate speculative accesses (see ARMv8 ARM G.a D5.2.9 for the details). And yes, implementations definitely do that. Add side-effect reads to the mix, and you're in for a treat. > We've been using this option internally for testing purpose and > validated with NVME/Mellanox/GPU pass-through devices on > Marvell-Thundex2 platform. See above. It *will* break eventually. > Option-2: Get resource properties associated with MMIO using lookup_resource() > and map at stage-2 as Normal-NC if IORESOURCE_PREFETCH is set in flags. That's a pretty roundabout way of doing exactly the same thing you initially proposed. And it suffers from the exact same problems, which is that you change the semantics of the mapping without knowing what the guest's intent is. M. -- Without deviation from the norm, progress is not possible.