From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AA7DC433DF for ; Tue, 28 Jul 2020 17:43:16 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6BBC62070B for ; Tue, 28 Jul 2020 17:43:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="Isvlt8g6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6BBC62070B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k0Tcs-0005jG-Jp; Tue, 28 Jul 2020 17:42:54 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k0Tcq-0005jB-Nc for xen-devel@lists.xenproject.org; Tue, 28 Jul 2020 17:42:52 +0000 X-Inumbo-ID: c1b1a2a6-d0f9-11ea-8ba9-bc764e2007e4 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id c1b1a2a6-d0f9-11ea-8ba9-bc764e2007e4; Tue, 28 Jul 2020 17:42:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1595958172; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=bm6xpS3UPlSHgrr/AwBHxbllr8iQgva7zYaEqmfywQE=; b=Isvlt8g6tTyBuPgT6CzTYbpm21V5W2Qq0D78W3YuJnZTzZvyM4MLtigM 92A2YgHALjRR9DT0V2nHkaxuWsIbgD2l1OuNmgFfhJbWeRuL/ZyD4tGSU NSEaH1ZCxWuEOkbFAYJ/LETnb6RZeE10gXtXtmiJMTYDDxrxbrtbZlhxg c=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: tqfc4/1PrubWMv5ewSP0LHdTujHFP8U+m86/9kdKUf1uh2GxSqdRLKvvIjL7M7hl/BUVdvDraQ 7xWgR+boyI9h5kdqmKdL4Un2zB12iUpSymGIbUEEQ7O8Hw5Bf2EayFOnHW8bDGV9iixxNVgLOC ohHW/uZUKSlMasSdB5PVjkQ2CjgPz8+aAB4j7j0KQdWkgHS/AF8C2gxIr9YIlD7LejhmX4/3f6 NuNWHo6AupAExmeZCKgns9LBWBBDJcckRANyYzThzC0R8PZACty0nlqgJXI7DofDVY/yjkEeEd sT0= X-SBRS: 2.7 X-MesageID: 23367204 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.75,406,1589256000"; d="scan'208";a="23367204" Date: Tue, 28 Jul 2020 19:42:42 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Andrew Cooper Subject: Re: [PATCH v3 4/4] xen: add helpers to allocate unpopulated memory Message-ID: <20200728174242.GB7191@Air-de-Roger> References: <20200727091342.52325-1-roger.pau@citrix.com> <20200727091342.52325-5-roger.pau@citrix.com> <20200728165919.GA7191@Air-de-Roger> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Oleksandr Andrushchenko , David Airlie , Yan Yankovskyi , David Hildenbrand , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Michal Hocko , linux-mm@kvack.org, Daniel Vetter , xen-devel@lists.xenproject.org, Boris Ostrovsky , Dan Williams , Dan Carpenter Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" On Tue, Jul 28, 2020 at 06:06:25PM +0100, Andrew Cooper wrote: > On 28/07/2020 17:59, Roger Pau Monné wrote: > > On Tue, Jul 28, 2020 at 05:48:23PM +0100, Julien Grall wrote: > >> Hi, > >> > >> On 27/07/2020 10:13, Roger Pau Monne wrote: > >>> To be used in order to create foreign mappings. This is based on the > >>> ZONE_DEVICE facility which is used by persistent memory devices in > >>> order to create struct pages and kernel virtual mappings for the IOMEM > >>> areas of such devices. Note that on kernels without support for > >>> ZONE_DEVICE Xen will fallback to use ballooned pages in order to > >>> create foreign mappings. > >>> > >>> The newly added helpers use the same parameters as the existing > >>> {alloc/free}_xenballooned_pages functions, which allows for in-place > >>> replacement of the callers. Once a memory region has been added to be > >>> used as scratch mapping space it will no longer be released, and pages > >>> returned are kept in a linked list. This allows to have a buffer of > >>> pages and prevents resorting to frequent additions and removals of > >>> regions. > >>> > >>> If enabled (because ZONE_DEVICE is supported) the usage of the new > >>> functionality untangles Xen balloon and RAM hotplug from the usage of > >>> unpopulated physical memory ranges to map foreign pages, which is the > >>> correct thing to do in order to avoid mappings of foreign pages depend > >>> on memory hotplug. > >> I think this is going to break Dom0 on Arm if the kernel has been built with > >> hotplug. This is because you may end up to re-use region that will be used > >> for the 1:1 mapping of a foreign map. > >> > >> Note that I don't know whether hotplug has been tested on Xen on Arm yet. So > >> it might be possible to be already broken. > >> > >> Meanwhile, my suggestion would be to make the use of hotplug in the balloon > >> code conditional (maybe using CONFIG_ARM64 and CONFIG_ARM)? > > Right, this feature (allocation of unpopulated memory separated from > > the balloon driver) is currently gated on CONFIG_ZONE_DEVICE, which I > > think could be used on Arm. > > > > IMO the right solution seems to be to subtract the physical memory > > regions that can be used for the identity mappings of foreign pages > > (all RAM on the system AFAICT) from iomem_resource, as that would make > > this and the memory hotplug done in the balloon driver safe? > > The right solution is a mechanism for translated guests to query Xen to > find regions of guest physical address space which are unused, and can > be safely be used for foreign/grant/other  mappings. > > Please don't waste any more time applying more duct tape to a broken > system, and instead spend the time organising some proper foundations. The piece added here (using ZONE_DEVICE) will be relevant when Xen can provide the space to map foreign pages, it's just that right now it relies on iomem_resource instead of a Xen specific resource map that should be provided by the hypervisor. It should indeed be fixed, but right now this patch should allow a PVH dom0 to work slightly better. When Xen provides such areas Linux just needs to populate a custom Xen resource with them and use it instead of iomem_resurce. The Arm stuff I'm certainly not familiar with, and can't provide much insight on that. If it's best to just disable it and continue to rely on ballooned out pages that's fine. Roger. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C485C433ED for ; Tue, 28 Jul 2020 19:31:22 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 32ED220656 for ; Tue, 28 Jul 2020 19:31:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="c68YZvbK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 32ED220656 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 23C3F6E39B; Tue, 28 Jul 2020 19:30:53 +0000 (UTC) Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com [216.71.145.155]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9EAFB6E21A for ; Tue, 28 Jul 2020 17:42:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1595958171; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=bm6xpS3UPlSHgrr/AwBHxbllr8iQgva7zYaEqmfywQE=; b=c68YZvbKM6yvoUYtaO8IeJCtT59yKXQfAbftBGBAgv5t8N64G6o8mzin lVSp8DnDsNOj/zhB9GuaLMIwhXuAx28UsPzTr1QyEhGfXWgOTEHvPytbb +ai8cfgaATyb/S1REt4RFvd7Enw9aFAL8gRMWzQFChhpu4bU8WEL+ZGih Q=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: tqfc4/1PrubWMv5ewSP0LHdTujHFP8U+m86/9kdKUf1uh2GxSqdRLKvvIjL7M7hl/BUVdvDraQ 7xWgR+boyI9h5kdqmKdL4Un2zB12iUpSymGIbUEEQ7O8Hw5Bf2EayFOnHW8bDGV9iixxNVgLOC ohHW/uZUKSlMasSdB5PVjkQ2CjgPz8+aAB4j7j0KQdWkgHS/AF8C2gxIr9YIlD7LejhmX4/3f6 NuNWHo6AupAExmeZCKgns9LBWBBDJcckRANyYzThzC0R8PZACty0nlqgJXI7DofDVY/yjkEeEd sT0= X-SBRS: 2.7 X-MesageID: 23367204 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.75,406,1589256000"; d="scan'208";a="23367204" Date: Tue, 28 Jul 2020 19:42:42 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Andrew Cooper Subject: Re: [PATCH v3 4/4] xen: add helpers to allocate unpopulated memory Message-ID: <20200728174242.GB7191@Air-de-Roger> References: <20200727091342.52325-1-roger.pau@citrix.com> <20200727091342.52325-5-roger.pau@citrix.com> <20200728165919.GA7191@Air-de-Roger> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-Mailman-Approved-At: Tue, 28 Jul 2020 19:30:50 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Oleksandr Andrushchenko , David Airlie , Yan Yankovskyi , David Hildenbrand , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Michal Hocko , linux-mm@kvack.org, xen-devel@lists.xenproject.org, Boris Ostrovsky , Dan Williams , Dan Carpenter Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" T24gVHVlLCBKdWwgMjgsIDIwMjAgYXQgMDY6MDY6MjVQTSArMDEwMCwgQW5kcmV3IENvb3BlciB3 cm90ZToKPiBPbiAyOC8wNy8yMDIwIDE3OjU5LCBSb2dlciBQYXUgTW9ubsOpIHdyb3RlOgo+ID4g T24gVHVlLCBKdWwgMjgsIDIwMjAgYXQgMDU6NDg6MjNQTSArMDEwMCwgSnVsaWVuIEdyYWxsIHdy b3RlOgo+ID4+IEhpLAo+ID4+Cj4gPj4gT24gMjcvMDcvMjAyMCAxMDoxMywgUm9nZXIgUGF1IE1v bm5lIHdyb3RlOgo+ID4+PiBUbyBiZSB1c2VkIGluIG9yZGVyIHRvIGNyZWF0ZSBmb3JlaWduIG1h cHBpbmdzLiBUaGlzIGlzIGJhc2VkIG9uIHRoZQo+ID4+PiBaT05FX0RFVklDRSBmYWNpbGl0eSB3 aGljaCBpcyB1c2VkIGJ5IHBlcnNpc3RlbnQgbWVtb3J5IGRldmljZXMgaW4KPiA+Pj4gb3JkZXIg dG8gY3JlYXRlIHN0cnVjdCBwYWdlcyBhbmQga2VybmVsIHZpcnR1YWwgbWFwcGluZ3MgZm9yIHRo ZSBJT01FTQo+ID4+PiBhcmVhcyBvZiBzdWNoIGRldmljZXMuIE5vdGUgdGhhdCBvbiBrZXJuZWxz IHdpdGhvdXQgc3VwcG9ydCBmb3IKPiA+Pj4gWk9ORV9ERVZJQ0UgWGVuIHdpbGwgZmFsbGJhY2sg dG8gdXNlIGJhbGxvb25lZCBwYWdlcyBpbiBvcmRlciB0bwo+ID4+PiBjcmVhdGUgZm9yZWlnbiBt YXBwaW5ncy4KPiA+Pj4KPiA+Pj4gVGhlIG5ld2x5IGFkZGVkIGhlbHBlcnMgdXNlIHRoZSBzYW1l IHBhcmFtZXRlcnMgYXMgdGhlIGV4aXN0aW5nCj4gPj4+IHthbGxvYy9mcmVlfV94ZW5iYWxsb29u ZWRfcGFnZXMgZnVuY3Rpb25zLCB3aGljaCBhbGxvd3MgZm9yIGluLXBsYWNlCj4gPj4+IHJlcGxh Y2VtZW50IG9mIHRoZSBjYWxsZXJzLiBPbmNlIGEgbWVtb3J5IHJlZ2lvbiBoYXMgYmVlbiBhZGRl ZCB0byBiZQo+ID4+PiB1c2VkIGFzIHNjcmF0Y2ggbWFwcGluZyBzcGFjZSBpdCB3aWxsIG5vIGxv bmdlciBiZSByZWxlYXNlZCwgYW5kIHBhZ2VzCj4gPj4+IHJldHVybmVkIGFyZSBrZXB0IGluIGEg bGlua2VkIGxpc3QuIFRoaXMgYWxsb3dzIHRvIGhhdmUgYSBidWZmZXIgb2YKPiA+Pj4gcGFnZXMg YW5kIHByZXZlbnRzIHJlc29ydGluZyB0byBmcmVxdWVudCBhZGRpdGlvbnMgYW5kIHJlbW92YWxz IG9mCj4gPj4+IHJlZ2lvbnMuCj4gPj4+Cj4gPj4+IElmIGVuYWJsZWQgKGJlY2F1c2UgWk9ORV9E RVZJQ0UgaXMgc3VwcG9ydGVkKSB0aGUgdXNhZ2Ugb2YgdGhlIG5ldwo+ID4+PiBmdW5jdGlvbmFs aXR5IHVudGFuZ2xlcyBYZW4gYmFsbG9vbiBhbmQgUkFNIGhvdHBsdWcgZnJvbSB0aGUgdXNhZ2Ug b2YKPiA+Pj4gdW5wb3B1bGF0ZWQgcGh5c2ljYWwgbWVtb3J5IHJhbmdlcyB0byBtYXAgZm9yZWln biBwYWdlcywgd2hpY2ggaXMgdGhlCj4gPj4+IGNvcnJlY3QgdGhpbmcgdG8gZG8gaW4gb3JkZXIg dG8gYXZvaWQgbWFwcGluZ3Mgb2YgZm9yZWlnbiBwYWdlcyBkZXBlbmQKPiA+Pj4gb24gbWVtb3J5 IGhvdHBsdWcuCj4gPj4gSSB0aGluayB0aGlzIGlzIGdvaW5nIHRvIGJyZWFrIERvbTAgb24gQXJt IGlmIHRoZSBrZXJuZWwgaGFzIGJlZW4gYnVpbHQgd2l0aAo+ID4+IGhvdHBsdWcuIFRoaXMgaXMg YmVjYXVzZSB5b3UgbWF5IGVuZCB1cCB0byByZS11c2UgcmVnaW9uIHRoYXQgd2lsbCBiZSB1c2Vk Cj4gPj4gZm9yIHRoZSAxOjEgbWFwcGluZyBvZiBhIGZvcmVpZ24gbWFwLgo+ID4+Cj4gPj4gTm90 ZSB0aGF0IEkgZG9uJ3Qga25vdyB3aGV0aGVyIGhvdHBsdWcgaGFzIGJlZW4gdGVzdGVkIG9uIFhl biBvbiBBcm0geWV0LiBTbwo+ID4+IGl0IG1pZ2h0IGJlIHBvc3NpYmxlIHRvIGJlIGFscmVhZHkg YnJva2VuLgo+ID4+Cj4gPj4gTWVhbndoaWxlLCBteSBzdWdnZXN0aW9uIHdvdWxkIGJlIHRvIG1h a2UgdGhlIHVzZSBvZiBob3RwbHVnIGluIHRoZSBiYWxsb29uCj4gPj4gY29kZSBjb25kaXRpb25h bCAobWF5YmUgdXNpbmcgQ09ORklHX0FSTTY0IGFuZCBDT05GSUdfQVJNKT8KPiA+IFJpZ2h0LCB0 aGlzIGZlYXR1cmUgKGFsbG9jYXRpb24gb2YgdW5wb3B1bGF0ZWQgbWVtb3J5IHNlcGFyYXRlZCBm cm9tCj4gPiB0aGUgYmFsbG9vbiBkcml2ZXIpIGlzIGN1cnJlbnRseSBnYXRlZCBvbiBDT05GSUdf Wk9ORV9ERVZJQ0UsIHdoaWNoIEkKPiA+IHRoaW5rIGNvdWxkIGJlIHVzZWQgb24gQXJtLgo+ID4K PiA+IElNTyB0aGUgcmlnaHQgc29sdXRpb24gc2VlbXMgdG8gYmUgdG8gc3VidHJhY3QgdGhlIHBo eXNpY2FsIG1lbW9yeQo+ID4gcmVnaW9ucyB0aGF0IGNhbiBiZSB1c2VkIGZvciB0aGUgaWRlbnRp dHkgbWFwcGluZ3Mgb2YgZm9yZWlnbiBwYWdlcwo+ID4gKGFsbCBSQU0gb24gdGhlIHN5c3RlbSBB RkFJQ1QpIGZyb20gaW9tZW1fcmVzb3VyY2UsIGFzIHRoYXQgd291bGQgbWFrZQo+ID4gdGhpcyBh bmQgdGhlIG1lbW9yeSBob3RwbHVnIGRvbmUgaW4gdGhlIGJhbGxvb24gZHJpdmVyIHNhZmU/Cj4g Cj4gVGhlIHJpZ2h0IHNvbHV0aW9uIGlzIGEgbWVjaGFuaXNtIGZvciB0cmFuc2xhdGVkIGd1ZXN0 cyB0byBxdWVyeSBYZW4gdG8KPiBmaW5kIHJlZ2lvbnMgb2YgZ3Vlc3QgcGh5c2ljYWwgYWRkcmVz cyBzcGFjZSB3aGljaCBhcmUgdW51c2VkLCBhbmQgY2FuCj4gYmUgc2FmZWx5IGJlIHVzZWQgZm9y IGZvcmVpZ24vZ3JhbnQvb3RoZXLCoCBtYXBwaW5ncy4KPiAKPiBQbGVhc2UgZG9uJ3Qgd2FzdGUg YW55IG1vcmUgdGltZSBhcHBseWluZyBtb3JlIGR1Y3QgdGFwZSB0byBhIGJyb2tlbgo+IHN5c3Rl bSwgYW5kIGluc3RlYWQgc3BlbmQgdGhlIHRpbWUgb3JnYW5pc2luZyBzb21lIHByb3BlciBmb3Vu ZGF0aW9ucy4KClRoZSBwaWVjZSBhZGRlZCBoZXJlICh1c2luZyBaT05FX0RFVklDRSkgd2lsbCBi ZSByZWxldmFudCB3aGVuIFhlbiBjYW4KcHJvdmlkZSB0aGUgc3BhY2UgdG8gbWFwIGZvcmVpZ24g cGFnZXMsIGl0J3MganVzdCB0aGF0IHJpZ2h0IG5vdyBpdApyZWxpZXMgb24gaW9tZW1fcmVzb3Vy Y2UgaW5zdGVhZCBvZiBhIFhlbiBzcGVjaWZpYyByZXNvdXJjZSBtYXAgdGhhdApzaG91bGQgYmUg cHJvdmlkZWQgYnkgdGhlIGh5cGVydmlzb3IuIEl0IHNob3VsZCBpbmRlZWQgYmUgZml4ZWQsIGJ1 dApyaWdodCBub3cgdGhpcyBwYXRjaCBzaG91bGQgYWxsb3cgYSBQVkggZG9tMCB0byB3b3JrIHNs aWdodGx5IGJldHRlci4KV2hlbiBYZW4gcHJvdmlkZXMgc3VjaCBhcmVhcyBMaW51eCBqdXN0IG5l ZWRzIHRvIHBvcHVsYXRlIGEgY3VzdG9tIFhlbgpyZXNvdXJjZSB3aXRoIHRoZW0gYW5kIHVzZSBp dCBpbnN0ZWFkIG9mIGlvbWVtX3Jlc3VyY2UuCgpUaGUgQXJtIHN0dWZmIEknbSBjZXJ0YWlubHkg bm90IGZhbWlsaWFyIHdpdGgsIGFuZCBjYW4ndCBwcm92aWRlIG11Y2gKaW5zaWdodCBvbiB0aGF0 LiBJZiBpdCdzIGJlc3QgdG8ganVzdCBkaXNhYmxlIGl0IGFuZCBjb250aW51ZSB0byByZWx5Cm9u IGJhbGxvb25lZCBvdXQgcGFnZXMgdGhhdCdzIGZpbmUuCgpSb2dlci4KX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApk cmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Au b3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 550FFC433E1 for ; Tue, 28 Jul 2020 17:42:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 191CD20672 for ; Tue, 28 Jul 2020 17:42:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="Isvlt8g6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 191CD20672 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A2E728D0006; Tue, 28 Jul 2020 13:42:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B8CF8D0002; Tue, 28 Jul 2020 13:42:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 880608D0006; Tue, 28 Jul 2020 13:42:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id 6EE718D0002 for ; Tue, 28 Jul 2020 13:42:52 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 289BC180AD815 for ; Tue, 28 Jul 2020 17:42:52 +0000 (UTC) X-FDA: 77088204984.14.pear04_2a04ffe26f6b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id F0A541822987A for ; Tue, 28 Jul 2020 17:42:51 +0000 (UTC) X-HE-Tag: pear04_2a04ffe26f6b X-Filterd-Recvd-Size: 6255 Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com [216.71.145.155]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Tue, 28 Jul 2020 17:42:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1595958172; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=bm6xpS3UPlSHgrr/AwBHxbllr8iQgva7zYaEqmfywQE=; b=Isvlt8g6tTyBuPgT6CzTYbpm21V5W2Qq0D78W3YuJnZTzZvyM4MLtigM 92A2YgHALjRR9DT0V2nHkaxuWsIbgD2l1OuNmgFfhJbWeRuL/ZyD4tGSU NSEaH1ZCxWuEOkbFAYJ/LETnb6RZeE10gXtXtmiJMTYDDxrxbrtbZlhxg c=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: tqfc4/1PrubWMv5ewSP0LHdTujHFP8U+m86/9kdKUf1uh2GxSqdRLKvvIjL7M7hl/BUVdvDraQ 7xWgR+boyI9h5kdqmKdL4Un2zB12iUpSymGIbUEEQ7O8Hw5Bf2EayFOnHW8bDGV9iixxNVgLOC ohHW/uZUKSlMasSdB5PVjkQ2CjgPz8+aAB4j7j0KQdWkgHS/AF8C2gxIr9YIlD7LejhmX4/3f6 NuNWHo6AupAExmeZCKgns9LBWBBDJcckRANyYzThzC0R8PZACty0nlqgJXI7DofDVY/yjkEeEd sT0= X-SBRS: 2.7 X-MesageID: 23367204 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.75,406,1589256000"; d="scan'208";a="23367204" Date: Tue, 28 Jul 2020 19:42:42 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Andrew Cooper CC: Julien Grall , Juergen Gross , "Stefano Stabellini" , Wei Liu , "Oleksandr Andrushchenko" , David Airlie , Yan Yankovskyi , David Hildenbrand , , , Michal Hocko , , Daniel Vetter , , Boris Ostrovsky , Dan Williams , "Dan Carpenter" Subject: Re: [PATCH v3 4/4] xen: add helpers to allocate unpopulated memory Message-ID: <20200728174242.GB7191@Air-de-Roger> References: <20200727091342.52325-1-roger.pau@citrix.com> <20200727091342.52325-5-roger.pau@citrix.com> <20200728165919.GA7191@Air-de-Roger> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-Rspamd-Queue-Id: F0A541822987A X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 28, 2020 at 06:06:25PM +0100, Andrew Cooper wrote: > On 28/07/2020 17:59, Roger Pau Monn=C3=A9 wrote: > > On Tue, Jul 28, 2020 at 05:48:23PM +0100, Julien Grall wrote: > >> Hi, > >> > >> On 27/07/2020 10:13, Roger Pau Monne wrote: > >>> To be used in order to create foreign mappings. This is based on th= e > >>> ZONE_DEVICE facility which is used by persistent memory devices in > >>> order to create struct pages and kernel virtual mappings for the IO= MEM > >>> areas of such devices. Note that on kernels without support for > >>> ZONE_DEVICE Xen will fallback to use ballooned pages in order to > >>> create foreign mappings. > >>> > >>> The newly added helpers use the same parameters as the existing > >>> {alloc/free}_xenballooned_pages functions, which allows for in-plac= e > >>> replacement of the callers. Once a memory region has been added to = be > >>> used as scratch mapping space it will no longer be released, and pa= ges > >>> returned are kept in a linked list. This allows to have a buffer of > >>> pages and prevents resorting to frequent additions and removals of > >>> regions. > >>> > >>> If enabled (because ZONE_DEVICE is supported) the usage of the new > >>> functionality untangles Xen balloon and RAM hotplug from the usage = of > >>> unpopulated physical memory ranges to map foreign pages, which is t= he > >>> correct thing to do in order to avoid mappings of foreign pages dep= end > >>> on memory hotplug. > >> I think this is going to break Dom0 on Arm if the kernel has been bu= ilt with > >> hotplug. This is because you may end up to re-use region that will b= e used > >> for the 1:1 mapping of a foreign map. > >> > >> Note that I don't know whether hotplug has been tested on Xen on Arm= yet. So > >> it might be possible to be already broken. > >> > >> Meanwhile, my suggestion would be to make the use of hotplug in the = balloon > >> code conditional (maybe using CONFIG_ARM64 and CONFIG_ARM)? > > Right, this feature (allocation of unpopulated memory separated from > > the balloon driver) is currently gated on CONFIG_ZONE_DEVICE, which I > > think could be used on Arm. > > > > IMO the right solution seems to be to subtract the physical memory > > regions that can be used for the identity mappings of foreign pages > > (all RAM on the system AFAICT) from iomem_resource, as that would mak= e > > this and the memory hotplug done in the balloon driver safe? >=20 > The right solution is a mechanism for translated guests to query Xen to > find regions of guest physical address space which are unused, and can > be safely be used for foreign/grant/other=C2=A0 mappings. >=20 > Please don't waste any more time applying more duct tape to a broken > system, and instead spend the time organising some proper foundations. The piece added here (using ZONE_DEVICE) will be relevant when Xen can provide the space to map foreign pages, it's just that right now it relies on iomem_resource instead of a Xen specific resource map that should be provided by the hypervisor. It should indeed be fixed, but right now this patch should allow a PVH dom0 to work slightly better. When Xen provides such areas Linux just needs to populate a custom Xen resource with them and use it instead of iomem_resurce. The Arm stuff I'm certainly not familiar with, and can't provide much insight on that. If it's best to just disable it and continue to rely on ballooned out pages that's fine. Roger.