From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9AA07C25B75 for ; Tue, 21 May 2024 11:49:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=CTh82chRGTkB2wQdBHlJvw9KurX9DrSYHzF+yZnGeTM=; b=C6IFzEzRU/YNIk 9ZPp5y+uaYfFx4LTmSpmnqZiLkf2Ssa5kCW44pz8mItBEVmD7gfZqnRMcLudZVNJFs5qrRbon0btX PNbd6LF4kGzhWDA5rlofJVaOhxa00NDLHwIuvLtFNxR6OeOYl+TvDnaXHsYlJHxeZ7L/UqV2PFlXi su3FZzru5AJi717k5P7M/9n6iFDCbobc1iqoxptO9MmFRvteiStp5FVwow2uZ+hrt1MdcyoKMc7Ix G1ie4i5IcoLcbSU+ej+XHBeeBZ50/ezwth2exzaEjOFnMKw0wN0hxLzgCPsZ3perz5G5CoPQl86Sm uhfQIrXQD6+myF7jDWjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s9NzF-0000000HKaH-1PV6; Tue, 21 May 2024 11:48:57 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s9Nz9-0000000HKWb-3FVo for linux-riscv@lists.infradead.org; Tue, 21 May 2024 11:48:54 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 99996CE0EB3; Tue, 21 May 2024 11:48:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 05DADC2BD11; Tue, 21 May 2024 11:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716292125; bh=w74oySSkmlGhE0TfH1tB7F1tx4YTKjzIh/pSxeIstk8=; h=From:To:Cc:Subject:Date:From; b=Z7Y2msQBI4viLlGJ0SpmMkYpvV3AS1iRehsdy1KRmCwEn7RV12UVquGp0XqkpYfNY v2aE1vjDmYa52XjWBtWrRfspwdE0jHGxK+PwYUvpFLIPi52dq2gHNp4JK79Az0x4Vl c9GvyfNDuZc9fnRcOAc2eza0B26LvqOJEZTA6tqIxXr0IKfjh3gc7Ry63ANlZxfqvE adAQpi119naNEOc24zn/tZAvOm++JqXSUymNnxV96y/k3esg9bWhzOiLbxRcuNBs5e odszu3eEZG+axxtfVKk9WsRbhNSkGYXY5+sARrQLydAe3LcYNsSrAFzJ4ITpn5qcE9 GJUHONG2kQL6Q== From: =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= To: Alexandre Ghiti , Albert Ou , David Hildenbrand , Palmer Dabbelt , Paul Walmsley , linux-riscv@lists.infradead.org, Oscar Salvador Subject: [PATCH v3 0/9] riscv: Memory Hot(Un)Plug support Date: Tue, 21 May 2024 13:48:21 +0200 Message-Id: <20240521114830.841660-1-bjorn@kernel.org> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240521_044852_829598_33F5BC29 X-CRM114-Status: GOOD ( 30.75 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lorenzo Stoakes , Chethan Seshadri , linux-kernel@vger.kernel.org, Andrew Bresticker , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Santosh Mamila , linux-mm@kvack.org, Sivakumar Munnangi , virtualization@lists.linux-foundation.org Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org RnJvbTogQmrDtnJuIFTDtnBlbCA8Ympvcm5Acml2b3NpbmMuY29tPgoKPT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQpNZW1vcnkg SG90KFVuKVBsdWcgc3VwcG9ydCAoYW5kIFpPTkVfREVWSUNFKSBmb3IgdGhlIFJJU0MtViBwb3J0 Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT0KCkludHJvZHVjdGlvbgo9PT09PT09PT09PT0KClRvIHF1b3RlICJEb2N1bWVudGF0 aW9uL2FkbWluLWd1aWRlL21tL21lbW9yeS1ob3RwbHVnLnJzdCI6ICJNZW1vcnkKaG90KHVuKXBs dWcgYWxsb3dzIGZvciBpbmNyZWFzaW5nIGFuZCBkZWNyZWFzaW5nIHRoZSBzaXplIG9mIHBoeXNp Y2FsCm1lbW9yeSBhdmFpbGFibGUgdG8gYSBtYWNoaW5lIGF0IHJ1bnRpbWUuIgoKVGhpcyBzZXJp ZXMgYWRkcyBtZW1vcnkgaG90KHVuKXBsdWdnaW5nLCBhbmQgWk9ORV9ERVZJQ0Ugc3VwcG9ydCBm b3IKdGhlIFJJU0MtViBMaW51eCBwb3J0LgoKTU0gY29uZmlndXJhdGlvbgo9PT09PT09PT09PT09 PT09CgpSSVNDLVYgTU0gaGFzIHRoZSBmb2xsb3dpbmcgY29uZmlndXJhdGlvbjoKCiAqIE1lbW9y eSBibG9ja3MgYXJlIDEyOE0sIGFuYWxvZ291cyB0byB4ODYtNjQuIEl0IHVzZXMgUE1ECiAgICgi aHVnZXBhZ2UiKSB2bWVtbWFwcy4gRnJvbSB0aGF0IGZvbGxvd3MgdGhhdCAyTSAoUE1EKSB3b3J0 aCBvZgogICB2bWVtbWFwIHNwYW5zIDMyNzY4IHBhZ2VzIMOhIDRLIHdoaWNoIGdldHMgdXMgMTI4 TS4KCiAqIFRoZSBwYWdlYmxvY2sgc2l6ZSBpcyB0aGUgbWluaW11bSBtaW5pbXVtIHZpcnRpb19t ZW0gc2l6ZSwgYW5kIG9uCiAgIFJJU0MtViBpdCdzIDJNICgyXjkgKiA0SykuCgpJbXBsZW1lbnRh dGlvbgo9PT09PT09PT09PT09PQoKVGhlIFBHRCB0YWJsZSBvbiBSSVNDLVYgaXMgc2hhcmVkL2Nv cGllZCBiZXR3ZWVuIGZvciBhbGwgcHJvY2Vzc2VzLiBUbwphdm9pZCBkb2luZyBwYWdlIHRhYmxl IHN5bmNocm9uaXphdGlvbiwgdGhlIGZpcnN0IHBhdGNoIChwYXRjaCAxKQpwcmUtYWxsb2NhdGVk IHRoZSBQR0QgZW50cmllcyBmb3Igdm1lbW1hcC9kaXJlY3QgbWFwLiBCeSBkb2luZyB0aGF0CnRo ZSBpbml0X21tIFBHRCB3aWxsIGJlIGZpeGVkIGF0IGtlcm5lbCBpbml0LCBhbmQgc3luY2hyb25p emF0aW9uIGNhbgpiZSBhdm9pZGVkIGFsbCB0b2dldGhlci4KClRoZSBmb2xsb3dpbmcgdHdvIHBh dGNoZXMgKHBhdGNoIDItMykgZG9lcyBzb21lIHByZXBhcmF0aW9ucywgZm9sbG93ZWQKYnkgdGhl IGFjdHVhbCBNSFAgaW1wbGVtZW50YXRpb24gKHBhdGNoIDQtNSkuIFRoZW4sIE1IUCBhbmQgdmly dGlvLW1lbQphcmUgZW5hYmxlZCAocGF0Y2ggNi03KSwgYW5kIGZpbmFsbHkgWk9ORV9ERVZJQ0Ug c3VwcG9ydCBpcyBhZGRlZAoocGF0Y2ggOCkuCgpNSFAgYW5kIGxvY2tpbmcKPT09PT09PT09PT09 PT09CgpUTDtEUjogVGhlIE1IUCBkb2VzIG5vdCBzdGVwIG9uIGFueSB0b2VzLCBleGNlcHQgZm9y IHB0ZHVtcC4KQWRkaXRpb25hbCBsb2NraW5nIGlzIHJlcXVpcmVkIGZvciBwdGR1bXAuCgpMb25n IHZlcnNpb246IEZvciB2MiBJIHNwZW50IHNvbWUgdGltZSBkaWdnaW5nIGludG8gaW5pdF9tbQpz eW5jaHJvbml6YXRpb24vdXBkYXRlLiBIZXJlIGFyZSBteSBmaW5kaW5ncywgYW5kIEknZCBsb3Zl IHRoZW0gdG8gYmUKY29ycmVjdGVkIGlmIGluY29ycmVjdC4KCkl0J3MgYmVlbiBhIGduYXJseSBw YXRoLi4uCgpUaGUgYGluaXRfbW1gIHN0cnVjdHVyZSBpcyBhIHNwZWNpYWwgbW0gKHBlcmhhcHMg bm90IGEgInJlYWwiIG9uZSkuCkl0J3MgYSAibGF6eSBjb250ZXh0IiB0aGF0IHRyYWNrcyBrZXJu ZWwgcGFnZSB0YWJsZSByZXNvdXJjZXMsIGUuZy4sCnRoZSBrZXJuZWwgcGFnZSB0YWJsZSAoc3dh cHBlcl9wZ19kaXIpLCBhIGtlcm5lbCBwYWdlX3RhYmxlX2xvY2sgKG1vcmUKYWJvdXQgdGhlIHVz YWdlIGJlbG93KSwgbW1hcF9sb2NrLCBhbmQgc3VjaC4KCmBpbml0X21tYCBkb2VzIG5vdCB0cmFj ay9jb250YWluIGFueSBWTUFzLiBIYXZpbmcgdGhlIGBpbml0X21tYCBpcwpjb252ZW5pZW50LCBz byB0aGF0IHRoZSByZWd1bGFyIGtlcm5lbCBwYWdlIHRhYmxlIHdhbGsvbW9kaWZ5CmZ1bmN0aW9u cyBjYW4gYmUgdXNlZC4KCk5vdywgYGluaXRfbW1gIGJlaW5nIHNwZWNpYWwgbWVhbnMgdGhhdCB0 aGUgbG9ja2luZyBmb3Iga2VybmVsIHBhZ2UKdGFibGVzIGFyZSBzcGVjaWFsIGFzIHdlbGwuCgpP biBSSVNDLVYgdGhlIFBHRCAodG9wLWxldmVsIHBhZ2UgdGFibGUgc3RydWN0dXJlKSwgc2ltaWxh ciB0byB4ODYsIGlzCnNoYXJlZCAoY29waWVkKSB3aXRoIHVzZXIgcHJvY2Vzc2VzLiBJZiB0aGUg a2VybmVsIFBHRCBpcyBtb2RpZmllZCwgaXQKaGFzIHRvIGJlIHN5bmNoZWQgdG8gdXNlci1tb2Rl IHByb2Nlc3NlcyBQR0RzLiBUaGlzIGlzIGF2b2lkZWQgYnkKcHJlLXBvcHVsYXRpbmcgdGhlIFBH RCwgc28gaXQnbGwgYmUgZml4ZWQgZnJvbSBib290LgoKVGhlIGluLWtlcm5lbCBwZ2QgcmVnaW9u cyBhcmUgZG9jdW1lbnRlZCBpbgpgRG9jdW1lbnRhdGlvbi9hcmNoL3Jpc2N2L3ZtLWxheW91dC5y c3RgLgoKVGhlIGRpc3RpbmN0IHJlZ2lvbnMgYXJlOgogKiB2bWVtbWFwCiAqIHZtYWxsb2MvaW9y ZW1hcCBzcGFjZQogKiBkaXJlY3QgbWFwcGluZyBvZiBhbGwgcGh5c2ljYWwgbWVtb3J5CiAqIGth c2FuCiAqIG1vZHVsZXMsIEJQRgogKiBrZXJuZWwKCk1lbW9yeSBob3RwbHVnIGlzIHRoZSBwcm9j ZXNzIG9mIGFkZGluZy9yZW1vdmluZyBtZW1vcnkgdG8vZnJvbSB0aGUKa2VybmVsLgoKQWRkaW5n IGlzIGRvbmUgaW4gdHdvIHBoYXNlczoKIDEuIEFkZCB0aGUgbWVtb3J5IHRvIHRoZSBrZXJuZWwK IDIuIE9ubGluZSBtZW1vcnksIG1ha2luZyBpdCBhdmFpbGFibGUgdG8gdGhlIHBhZ2UgYWxsb2Nh dG9yLgoKU3RlcCAxIGlzIHBhcnRpYWxseSBhcmNoaXRlY3R1cmUgZGVwZW5kZW50LCBhbmQgdXBk YXRlcyB0aGUgaW5pdF9tbQpwYWdlIHRhYmxlOgogKiBVcGRhdGUgdGhlIGRpcmVjdCBtYXAgcGFn ZSB0YWJsZXMuIFRoZSBkaXJlY3QgbWFwIGlzIGEgbGluZWFyIG1hcCwKICAgcmVwcmVzZW50aW5n IGFsbCBwaHlzaWNhbCBtZW1vcnk6IGB2aXJ0ID0gcGh5cyArIFBBR0VfT0ZGU0VUYAogKiBBZGQg YSBgc3RydWN0IHBhZ2VgIGZvciBlYWNoIGFkZGVkIHBhZ2Ugb2YgbWVtb3J5LiBVcGRhdGUgdGhl CiAgIHZtZW1tYXAgKHZpcnR1YWwgbWFwcGluZyB0byB0aGUgYHN0cnVjdCBwYWdlYCwgc28gd2Ug Y2FuIGVhc2lseQogICB0cmFuc2Zvcm0gYSBrZXJuZWwgdmlydHVhbCBhZGRyZXNzIHRvIGEgYHN0 cnVjdCBwYWdlICpgIGFkZHJlc3MuCgpGcm9tIGFuIE1IUCBwZXJzcGVjdGl2ZSwgdGhlcmUgYXJl IHR3byByZWdpb25zIG9mIHRoZSBQR0QgdGhhdCBhcmUKdXBkYXRlZDoKICogdm1lbW1hcAogKiBk aXJlY3QgbWFwcGluZyBvZiBhbGwgcGh5c2ljYWwgbWVtb3J5CgpUaGUgYHN0cnVjdCBtbV9zdHJ1 Y3RgIGhhcyBhIGNvdXBsZSBvZiBsb2NrcyBpbiBwbGF5OgogKiBgc3BpbmxvY2tfdCBwYWdlX3Rh YmxlX2xvY2tgIHByb3RlY3RzIHRoZSBwYWdlIHRhYmxlLCBhbmQgc29tZQogICAgY291bnRlcnMK ICogYHN0cnVjdCByd19zZW1hcGhvcmUgbW1hcF9sb2NrYCBwcm90ZWN0IGFuIG1tJ3MgVk1BcwoK Tm90ZSBhZ2FpbiB0aGF0IGBpbml0X21tYCBkb2VzIG5vdCBjb250YWluIGFueSBWTUFzLCBidXQg c3RpbGwgdXNlcwp0aGUgbW1hcF9sb2NrIGluIHNvbWUgcGxhY2VzLgoKVGhlIGBwYWdlX3RhYmxl X2xvY2tgIHdhcyBvcmlnaW5hbGx5IHVzZWQgdG8gdG8gcHJvdGVjdCBhbGwgcGFnZXMKdGFibGVz LCBidXQgbW9yZSByZWNlbnRseSBhIHNwbGl0IHBhZ2UgdGFibGUgbG9jayBoYXMgYmVlbiBpbnRy b2R1Y2VkLgpUaGUgc3BsaXQgbG9jayBoYXMgYSBwZXItdGFibGUgbG9jayBmb3IgdGhlIFBURSBh bmQgUE1EIHRhYmxlcy4gSWYKc3BsaXQgbG9jayBpcyBkaXNhYmxlZCwgYWxsIHRhYmxlcyBhcmUg Z3VhcmRlZCBieQpgbW0tPnBhZ2VfdGFibGVfbG9ja2AgKGZvciB1c2VyIHByb2Nlc3NlcykuIFNw bGl0IHBhZ2UgdGFibGUgbG9ja3MgYXJlCm5vdCB1c2VkIGZvciBpbml0X21tLgoKTUhQIG9wZXJh dGlvbnMgaXMgdHlwaWNhbGx5IHN5bmNocm9uaXplZCB1c2luZwpgREVGSU5FX1NUQVRJQ19QRVJD UFVfUldTRU0obWVtX2hvdHBsdWdfbG9jaylgLgoKQWN0b3JzCi0tLS0tLQoKVGhlIGZvbGxvd2lu ZyBub24tTUhQIGFjdG9ycyBpbiB0aGUga2VybmVsIHRyYXZlcnNlcyAocmVhZCksIGFuZC9vcgpt b2RpZmllcyB0aGUga2VybmVsIFBHRC4KCiAqIGBwdGR1bXBgCgogICBXYWxrcyB0aGUgZW50aXJl IGBpbml0X21tYCwgdmlhIGBwdGR1bXBfd2Fsa19wZ2QoKWAgd2l0aCB0aGUKICAgYG1tYXBfd3Jp dGVfbG9jayhpbml0X21tKWAgdGFrZW4uCgogICBPYnNlcnZhdGlvbjogcHRkdW1wIGNhbiByYWNl IHdpdGggTUhQLCBhbmQgbmVlZHMgYWRkaXRpb25hbCBsb2NraW5nCiAgIHRvIGF2b2lkIGNyYXNo ZXMvcmFjZXMuCgogKiBgc2V0X2RpcmVjdF8qYCAvIGBhcmNoL3Jpc2N2L21tL3BhZ2VhdHRyLmNg CgogICBUaGUgYHNldF9kaXJlY3RfKmAgZnVuY3Rpb25hbGl0eSBpcyB1c2VkIHRvICJzeW5jaHJv bml6ZSIgdGhlCiAgIGRpcmVjdCBtYXAgdG8gb3RoZXIga2VybmVsIG1hcHBpbmdzLCBlLmcuIG1v ZHVsZXMva2VybmVsIHRleHQuIFRoZQogICBkaXJlY3QgbWFwIGlzIHVzaW5nICJhcyBsYXJnZSBo dWdlIHRhYmxlIG1hcHBpbmdzIGFzIHBvc3NpYmxlIiwKICAgd2hpY2ggbWVhbnMgdGhhdCB0aGUg YHNldF9kaXJlY3RfKmAgbWlnaHQgbmVlZCB0byBzcGxpdCB0aGUgZGlyZWN0CiAgIG1hcC4KCiAg VGhlIGBzZXRfZGlyZWN0XypgIGZ1bmN0aW9ucyBvcGVyYXRlcyB3aXRoIHRoZQogIGBtbWFwX3dy aXRlX2xvY2soaW5pdF9tbSlgIHRha2VuLgoKICBPYnNlcnZhdGlvbjogYHNldF9kaXJlY3RfKmAg dXNlcyB0aGUgZGlyZWN0IG1hcCwgYnV0IHdpbGwgbmV2ZXIKICBtb2RpZnkgdGhlIHNhbWUgZW50 cnkgYXMgTUhQLiBJZiB0aGVyZSBpcyBhIG1hcHBpbmcsIHRoYXQgZW50cnkgd2lsbAogIG5ldmVy IHJhY2Ugd2l0aCBNSFAuIEZ1cnRoZXIsIE1IUCBhY3RzIHdoZW4gbWVtb3J5IGlzIG9mZmxpbmUu CgogKiBIVk8gLyBgbW0vaHVnZXRsYl92bWVtbWFwYAoKICAgSFZPIG9wdGltaXplcyB0aGUgYmFj a2luZyBgc3RydWN0IHBhZ2VgIGZvciBodWdldGxiIHBhZ2VzLCB3aGljaAogICBtZWFucyBjaGFu Z2luZyB0aGUgInZtZW1tYXAiIHJlZ2lvbi4gSFZPIGNhbiBzcGxpdCAobWVyZ2U/KSBhCiAgIHZt ZW1tYXAgcG1kLiBIb3dldmVyLCBpdCB3aWxsIG5ldmVyIHJhY2Ugd2l0aCBNSFAsIHNpbmNlIEhW TyBvbmx5CiAgIG9wZXJhdGVzIGF0IG9ubGluZSBtZW1vcnkuIEhWTyBjYW5ub3QgdG91Y2ggbWVt b3J5IGJlaW5nIE1IUCBhZGRlZAogICBvciByZW1vdmVkLgoKICogYGFwcGx5X3RvX3BhZ2VfcmFu Z2VgCgogICBXYWxrcyBhIHJhbmdlLCBjcmVhdGVzIHBhZ2VzIGFuZCBhcHBsaWVzIGEgY2FsbGJh Y2sgKHNldHRpbmcKICAgcGVybWlzc2lvbnMpIGZvciB0aGUgcGFnZS4KCiAgIFdoZW4gY3JlYXRp bmcgYSB0YWJsZSwgaXQgbWlnaHQgdXNlIGBpbnQgX19wdGVfYWxsb2Nfa2VybmVsKHBtZF90CiAg ICpwbWQpYCB3aGljaCB0YWtlcyB0aGUgYGluaXRfbW0ucGFnZV90YWJsZV9sb2NrYCB0byBzeW5j aHJvbml6ZSBwbWQKICAgcG9wdWxhdGUuCgogICBVc2VkIGJ5OiBgbW0vdm1hbGxvYy5jYCBhbmQg YG1tL2thc2FuL3NoYWRvdy5jYC4gVGhlIEtBU0FOIGNhbGxiYWNrCiAgIHRha2VzIHRoZSBgaW5p dF9tbS5wYWdlX3RhYmxlX2xvY2tgIHRvIHN5bmNocm9uaXplIHB0ZSBjcmVhdGlvbi4KCiAgIE9i c2VydmF0aW9uczogYGFwcGx5X3RvX3BhZ2VfcmFuZ2VgIGFwcGxpZXMgdG8gdGhlICJ2bWFsbG9j L2lvcmVtYXAKICAgc3BhY2UiIHJlZ2lvbiwgYW5kICJrYXNhbiIgcmVnaW9uLiAqTm90KiBhZmZl Y3RlZCBieSBNSFAuCgogKiBgYXBwbHlfdG9fZXhpc3RpbmdfcGFnZV9yYW5nZWAKCiAgIFdhbGtz IGEgcmFuZ2UsIGFwcGxpZXMgYSBjYWxsYmFjayAoc2V0dGluZyBwZXJtaXNzaW9ucykgZm9yIHRo ZQogICBwYWdlIChubyBwYWdlIGNyZWF0aW9uKS4KCiAgIFVzZWQgYnk6IGBrZXJuZWwvYnBmL2Fy ZW5hLmNgIGFuZCBgbW0va2FzYW4vc2hhZG93LmNgLiBUaGUgS0FTQU4KICAgY2FsbGJhY2sgdGFr ZXMgdGhlIGBpbml0X21tLnBhZ2VfdGFibGVfbG9ja2AgdG8gc3luY2hyb25pemUgcHRlCiAgIGNy ZWF0aW9uLiAqTm90KiBhZmZlY3RlZCBieSBNSFAgcmVnaW9ucy4KCiAqIGBhcHBseV90b19leGlz dGluZ19wYWdlX3JhbmdlYCBhcHBsaWVzIHRvIHRoZSAidm1hbGxvYy9pb3JlbWFwCiAgIHNwYWNl IiByZWdpb24sIGFuZCAia2FzYW4iIHJlZ2lvbi4gKk5vdCogYWZmZWN0ZWQgYnkgTUhQIHJlZ2lv bnMuCgogKiAgYGlvcmVtYXBfcGFnZV9yYW5nZWAgYW5kIGB2bWFwX3BhZ2VfcmFuZ2VgCgogICAg VXNlcyB0aGUgc2FtZSBpbnRlcm5hbCBmdW5jdGlvbiwgYW5kIG1pZ2h0IGNyZWF0ZSB0YWJsZSBl bnRyaWVzIGF0CiAgICB0aGUgInZtYWxsb2MvaW9yZW1hcCBzcGFjZSIgcmVnaW9uLiBDYW4gY2Fs bAogICAgYF9fcHRlX2FsbG9jX2tlcm5lbCgpYCB3aGljaCB0YWtlcyB0aGUgYGluaXRfbW0ucGFn ZV90YWJsZV9sb2NrYAogICAgc3luY2hyb25pemluZyBwbWQgcG9wdWxhdGUgaW4gdGhlIHJlZ2lv bi4gKk5vdCogYWZmZWN0ZWQgYnkgTUhQCiAgICByZWdpb25zLgoKU3VtbWFyeToKICAqIE1IUCBh ZGQgd2lsbCBuZXZlciBtb2RpZnkgdGhlIHNhbWUgcGFnZSB0YWJsZSBlbnRyaWVzLCBhcyBhbnkg b2YKICAgIHRoZSBvdGhlciBhY3RvcnMuCiAgKiBNSFAgcmVtb3ZlIGlzIGRvbmUgd2hlbiBtZW1v cnkgaXMgb2ZmbGluZWQsIGFuZCB3aWxsIG5vdCBjbGFzaAogICAgd2l0aCBhbnkgb2YgdGhlIGFj dG9ycy4KICAqIEZ1bmN0aW9ucyB0aGF0IHdhbGsgdGhlIGVudGlyZSBrZXJuZWwgcGFnZSB0YWJs ZSBuZWVkCiAgICBzeW5jaHJvbml6YXRpb24KCiAgKiBJdCdzIHN1ZmZpY2llbnQgdG8gYWRkIHRo ZSBNSFAgbG9jayBwdGR1bXAuCgpUZXN0aW5nCj09PT09PT0KClRoaXMgc2VyaWVzIGFkZHMgYmFz aWMgRFQgc3VwcG9ydGVkIGhvdHBsdWdnaW5nLiBUaGVyZSBpcyBhIFFFTVUKc2VyaWVzIGVuYWJs aW5nIE1IUCBmb3IgdGhlIFJJU0MtViAidmlydCIgbWFjaGluZSBoZXJlOiBbMV0KCkFDUEkvTVNJ IHN1cHBvcnQgaXMgc3RpbGwgaW4gdGhlIG1ha2luZyBmb3IgUklTQy1WLCBhbmQgcHJpb3IgcHJv cGVyCihBQ1BJKSBQQ0kgTVNJIHN1cHBvcnQgbGFuZHMgWzJdIGFuZCBOVU1BIFNSQVQgc3VwcG9y dCBbM10sIGl0IGhhcmQgdG8KdHJ5IGl0IG91dC4KCkkndmUgcHJlcGFyZWQgYSBRRU1VIGJyYW5j aCB3aXRoIHByb3BlciBBQ1BJIEdFRC9QQy1ESU1NIHN1cHBvcnQgWzRdLAphbmQgYSB0aGlzIHNl cmllcyB3aXRoIHRoZSByZXF1aXJlZCBwcmVyZXF1aXNpdGVzIFs1XSAoQUlBLCBBQ1BJIEFJQQpN QURULCBBQ1BJIE5VTUEgU1JBVCkuCgoKVG8gdGVzdCB3aXRoIHZpcnRpby1tZW0sIGUuZy46CiAg fCBxZW11LXN5c3RlbS1yaXNjdjY0IFwKICB8ICAgICAtbWFjaGluZSB2aXJ0LGFpYT1hcGxpYy1p bXNpYyBcCiAgfCAgICAgLWNwdSBydjY0LHY9dHJ1ZSx2bGVuPTI1NixlbGVuPTY0LGg9dHJ1ZSx6 YmtiPW9uLHpia2M9b24semJreD1vbix6a3I9b24semt0PW9uLHN2aW52YWw9b24sc3ZuYXBvdD1v bixzdnBibXQ9b24gXAogIHwgICAgIC1ub2RlZmF1bHRzIFwKICB8ICAgICAtbm9ncmFwaGljIC1z bXAgOCAta2VybmVsIHJ2NjQtdS1ib290LmJpbiBcCiAgfCAgICAgLWRyaXZlIGZpbGU9cm9vdGZz LmltZyxmb3JtYXQ9cmF3LGlmPXZpcnRpbyBcCiAgfCAgICAgLWRldmljZSB2aXJ0aW8tcm5nLXBj aSBcCiAgfCAgICAgLW0gMTZHLHNsb3RzPTMsbWF4bWVtPTMyRyBcCiAgfCAgICAgLW9iamVjdCBt ZW1vcnktYmFja2VuZC1yYW0saWQ9bWVtMCxzaXplPTE2RyBcCiAgfCAgICAgLW51bWEgbm9kZSxu b2RlaWQ9MCxtZW1kZXY9bWVtMCBcCiAgfCAgICAgLXNlcmlhbCBjaGFyZGV2OmNoYXIwIFwKICB8 ICAgICAtbW9uIGNoYXJkZXY9Y2hhcjAsbW9kZT1yZWFkbGluZSBcCiAgfCAgICAgLWNoYXJkZXYg c3RkaW8sbXV4PW9uLGlkPWNoYXIwIFwKICB8ICAgICAtZGV2aWNlIHBjaS1zZXJpYWwsaWQ9c2Vy aWFsMCxjaGFyZGV2PWNoYXIwIFwKICB8ICAgICAtb2JqZWN0IG1lbW9yeS1iYWNrZW5kLXJhbSxp ZD12bWVtMCxzaXplPTJHIFwKICB8ICAgICAtZGV2aWNlIHZpcnRpby1tZW0tcGNpLGlkPXZtMCxt ZW1kZXY9dm1lbTAsbm9kZT0wCgp3aGVyZSAicnY2NC11LWJvb3QuYmluIiBpcyBVLWJvb3Qgd2l0 aCBFRkkvQUNQSS1zdXBwb3J0ICh1c2UgWzZdIGlmCnlvdSdyZSBsYXp5KS4KCkluIHRoZSBRRU1V IG1vbml0b3I6CiAgfCAocWVtdSkgaW5mbyBtZW1vcnktZGV2aWNlcwogIHwgKHFlbXUpIHFvbS1z ZXQgdm0wIHJlcXVlc3RlZC1zaXplIDFHCgouLi50byB0ZXN0IERBWC9LTUVNLCB1c2UgdGhlIGZv bGxvdyBRRU1VIHBhcmFtZXRlcnM6CiAgfCAgLW9iamVjdCBtZW1vcnktYmFja2VuZC1maWxlLGlk PW1lbTEsc2hhcmU9b24sbWVtLXBhdGg9dmlydGlvX3BtZW0uaW1nLHNpemU9NEcgXAogIHwgIC1k ZXZpY2UgdmlydGlvLXBtZW0tcGNpLG1lbWRldj1tZW0xLGlkPW52MQoKYW5kIHRoZSByZWd1bGFy IG5kY3RsL2RheGN0bCBkYW5jZS4KCklmIHlvdSdyZSBicmF2ZSB0byB0cnkgdGhlIEFDUEkgYnJh bmNoLCBhZGQgImFjcGk9b24iIHRvICItbWFjaGluZQp2aXJ0IiwgYW5kIHRlc3QgUEMtRElNTSBN SFAgKGluIGFkZGl0aW9uIHRvIHZpcnRpby17cH0sbWVtKToKCkluIHRoZSBRRU1VIG1vbml0b3I6 CiAgfCAocWVtdSkgb2JqZWN0X2FkZCBtZW1vcnktYmFja2VuZC1yYW0saWQ9bWVtMSxzaXplPTFH CiAgfCAocWVtdSkgZGV2aWNlX2FkZCBwYy1kaW1tLGlkPWRpbW0xLG1lbWRldj1tZW0xCgpZb3Ug Y2FuIGFsc28gdHJ5IGhvdC1yZW1vdmUgd2l0aCBzb21lIFFFTVUgb3B0aW9ucywgc2F5OgogIHwg LW9iamVjdCBtZW1vcnktYmFja2VuZC1maWxlLGlkPW1lbS0xLHNpemU9MjU2TSxtZW0tcGF0aD0v cGFnZXNpemUtMk1CCiAgfCAtZGV2aWNlIHBjLWRpbW0saWQ9bWVtMSxtZW1kZXY9bWVtLTEKICB8 IC1vYmplY3QgbWVtb3J5LWJhY2tlbmQtZmlsZSxpZD1tZW0tMixzaXplPTFHLG1lbS1wYXRoPS9w YWdlc2l6ZS0xR0IKICB8IC1kZXZpY2UgcGMtZGltbSxpZD1tZW0yLG1lbWRldj1tZW0tMgogIHwg LW9iamVjdCBtZW1vcnktYmFja2VuZC1maWxlLGlkPW1lbS0zLHNpemU9MjU2TSxtZW0tcGF0aD0v cGFnZXNpemUtMk1CCiAgfCAtZGV2aWNlIHBjLWRpbW0saWQ9bWVtMyxtZW1kZXY9bWVtLTMKClJl bW92ZSAiYWNwaT1vbiIgdG8gcnVuIHdpdGggRFQuCgpUaGFua3MgdG8gQWxleCwgQW5kcmV3LCBE YXZpZCwgYW5kIE9zY2FyIGZvciBhbGwKY29tbWVudHMvdGVzdHMvZml4dXBzLgoKQ2hhbmdlbG9n Cj09PT09PT09PQoKdjItPnYzOgogIENvbGxlY3QgdGFncwogIDAwMDEtcmlzY3YtbW0tUHJvcGVy bHktZm9yd2FyZC12bWVtbWFwX3BvcHVsYXRlLWFsdG1hcAogICAgKiBOZXcgcGF0Y2gsIHNwbGl0 IGZyb20gYXR0cmlidXRlIGNoYW5nZSAoQWxleCwgT3NjYXIpCiAgMDAwNS1yaXNjdi1tbS1BZGQt bWVtb3J5LWhvdHBsdWdnaW5nLXN1cHBvcnQKICAgICogRml4IHBhZ2UgdGFibGUgZnJlZSB0byB0 YWtlIFBHX3Jlc2VydmVkIGludG8gY29uc2lkZXJhdGlvbiAoRGF2aWQsCiAgICAgIE9zY2FyKQog ICAgKiBEZXBlbmQgb24gU1BBUlNFX1ZNRU1NQVAgKEFsZXgsIERhdmlkLCBPc2NhcikKICAgICog UHJvcGVyIFRMQiBmbHVzaGluZyAoQWxleCkKICAwMDA5LXJpc2N2LW1tLUFkZC1zdXBwb3J0LWZv ci1aT05FX0RFVklDRQogICAgKiBGaXhlZCAzMmIgYnVpbGQKCnYxLT52MjoKICBSZXZpZXdlZCBh IGxvdCBvZiBNSFAgbG9ja2luZyBzY2VuYXJpb3MKICBWYXJpb3VzIGNvbmZpZyBidWlsZCBpc3N1 ZXMgKGUuZy4gYnVpbGQgIUNPTkZJR19NRU1PUllfSE9UUExVRykgKEFuZHJldykKICBBZGRlZCBh cmNoX2dldF9tYXBwYWJsZV9yYW5nZSgpIGltcGxlbWVudGF0aW9uCiAgQWNxdWlyZSBNSFAgbG9j ayBmb3IgcHRkdW1wLCBhbmFsb2dvdXMgdG8gYXJtNjQKICBBQ1BJIE1IUCB0ZXN0cwogIEFkZCBa T05FX0RFVklDRSBwYXRjaAoKUmVmZXJlbmNlcwo9PT09PT09PT09CgpbMV0gaHR0cHM6Ly9sb3Jl Lmtlcm5lbC5vcmcvcWVtdS1kZXZlbC8yMDI0MDUyMTEwNTYzNS43OTUyMTEtMS1iam9ybkBrZXJu ZWwub3JnLwpbMl0gaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvbGludXgtcmlzY3YvMjAyNDA1MDEx MjE3NDIuMTIxNTc5Mi0xLXN1bmlsdmxAdmVudGFuYW1pY3JvLmNvbS8KWzNdIGh0dHBzOi8vbG9y ZS5rZXJuZWwub3JnL2xpbnV4LXJpc2N2L2NvdmVyLjE3MTM3NzgyMzYuZ2l0LmhhaWJvMS54dUBp bnRlbC5jb20vCls0XSBodHRwczovL2dpdGh1Yi5jb20vYmpvdG8vcWVtdS9jb21taXRzL3ZpcnRp by1tZW0tcGMtZGltbS1taHAtYWNwaS12Mi8KWzVdIGh0dHBzOi8vZ2l0aHViLmNvbS9iam90by9s aW51eC9jb21taXRzL21ocC12My1uZXh0LWFjcGkKWzZdIGh0dHBzOi8vZ2l0aHViLmNvbS9iam90 by9yaXNjdi1yb290ZnMtdXRpbHMvdHJlZS9hY3BpCgoKVGhhbmtzLApCasO2cm4KCkJqw7ZybiBU w7ZwZWwgKDkpOgogIHJpc2N2OiBtbTogUHJvcGVybHkgZm9yd2FyZCB2bWVtbWFwX3BvcHVsYXRl KCkgYWx0bWFwIHBhcmFtZXRlcgogIHJpc2N2OiBtbTogUHJlLWFsbG9jYXRlIHZtZW1tYXAvZGly ZWN0IG1hcCBQR0QgZW50cmllcwogIHJpc2N2OiBtbTogQ2hhbmdlIGF0dHJpYnV0ZSBmcm9tIF9f aW5pdCB0byBfX21lbWluaXQgZm9yIHBhZ2UKICAgIGZ1bmN0aW9ucwogIHJpc2N2OiBtbTogUmVm YWN0b3IgY3JlYXRlX2xpbmVhcl9tYXBwaW5nX3JhbmdlKCkgZm9yIG1lbW9yeSBob3QgYWRkCiAg cmlzY3Y6IG1tOiBBZGQgbWVtb3J5IGhvdHBsdWdnaW5nIHN1cHBvcnQKICByaXNjdjogbW06IFRh a2UgbWVtb3J5IGhvdHBsdWcgcmVhZC1sb2NrIGR1cmluZyBrZXJuZWwgcGFnZSB0YWJsZSBkdW1w CiAgcmlzY3Y6IEVuYWJsZSBtZW1vcnkgaG90cGx1Z2dpbmcgZm9yIFJJU0MtVgogIHZpcnRpby1t ZW06IEVuYWJsZSB2aXJ0aW8tbWVtIGZvciBSSVNDLVYKICByaXNjdjogbW06IEFkZCBzdXBwb3J0 IGZvciBaT05FX0RFVklDRQoKIGFyY2gvcmlzY3YvS2NvbmZpZyAgICAgICAgICAgICAgICAgICAg fCAgIDMgKwogYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9rYXNhbi5oICAgICAgICB8ICAgNCArLQog YXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9tbXUuaCAgICAgICAgICB8ICAgNCArLQogYXJjaC9yaXNj di9pbmNsdWRlL2FzbS9wZ3RhYmxlLTY0LmggICB8ICAyMCArKwogYXJjaC9yaXNjdi9pbmNsdWRl L2FzbS9wZ3RhYmxlLWJpdHMuaCB8ICAgMSArCiBhcmNoL3Jpc2N2L2luY2x1ZGUvYXNtL3BndGFi bGUuaCAgICAgIHwgIDE5ICstCiBhcmNoL3Jpc2N2L21tL2luaXQuYyAgICAgICAgICAgICAgICAg IHwgMzM5ICsrKysrKysrKysrKysrKysrKysrKysrLS0tCiBhcmNoL3Jpc2N2L21tL3B0ZHVtcC5j ICAgICAgICAgICAgICAgIHwgICAzICsKIGRyaXZlcnMvdmlydGlvL0tjb25maWcgICAgICAgICAg ICAgICAgfCAgIDIgKy0KIDkgZmlsZXMgY2hhbmdlZCwgMzQ5IGluc2VydGlvbnMoKyksIDQ2IGRl bGV0aW9ucygtKQoKCmJhc2UtY29tbWl0OiA5MmNjZTkxOTQ5YTQ5N2E4YTQ2MTVmOWJhNTgxM2Iw M2Y3YTFmMWQ1Ci0tIAoyLjQwLjEKCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fXwpsaW51eC1yaXNjdiBtYWlsaW5nIGxpc3QKbGludXgtcmlzY3ZAbGlzdHMu aW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZv L2xpbnV4LXJpc2N2Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1AFC770ED for ; Tue, 21 May 2024 11:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=140.211.166.138 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716292134; cv=none; b=qtNtz/LoVCHXFX7LcaSpdpUp6kuJDGuq67Qpr91XEHy5DAbZM54urqDtCqm7ZihiIrAjLBUilE9J5oRdE5xIPI5gdin4iw9Hf6hfLspK4DQseiMJErFHkN6xWNmF5fM0uRbLOSJT3gop4rDRG8o8QnKW3z40SQcc1j0yb16oFN8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716292134; c=relaxed/simple; bh=w74oySSkmlGhE0TfH1tB7F1tx4YTKjzIh/pSxeIstk8=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type; b=nv3khV7pGczwcZVtzgNZe19qazROlfi6kxk5C/RuS+7JAH6H0PTHLGtx90UxOI5rlfKYCyrLNd3O7qOmln76PCdwCHNaLT+7pjrO/VF+ZNyWU/CBh41vUz8mAJOYeehXLcCKb7kkOKKgCy81BCQHVGZNWMrHW+vMjSvNFsJCH4U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Z7Y2msQB; arc=none smtp.client-ip=140.211.166.138 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Z7Y2msQB" Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5C0B980C41 for ; Tue, 21 May 2024 11:48:52 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org X-Spam-Flag: NO X-Spam-Score: -4.401 X-Spam-Level: Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id mLucOkMdmFCw for ; Tue, 21 May 2024 11:48:50 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=145.40.73.55; helo=sin.source.kernel.org; envelope-from=bjorn@kernel.org; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 1CB0180BE7 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=kernel.org DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1CB0180BE7 Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=Z7Y2msQB Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1CB0180BE7 for ; Tue, 21 May 2024 11:48:49 +0000 (UTC) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 99996CE0EB3; Tue, 21 May 2024 11:48:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 05DADC2BD11; Tue, 21 May 2024 11:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716292125; bh=w74oySSkmlGhE0TfH1tB7F1tx4YTKjzIh/pSxeIstk8=; h=From:To:Cc:Subject:Date:From; b=Z7Y2msQBI4viLlGJ0SpmMkYpvV3AS1iRehsdy1KRmCwEn7RV12UVquGp0XqkpYfNY v2aE1vjDmYa52XjWBtWrRfspwdE0jHGxK+PwYUvpFLIPi52dq2gHNp4JK79Az0x4Vl c9GvyfNDuZc9fnRcOAc2eza0B26LvqOJEZTA6tqIxXr0IKfjh3gc7Ry63ANlZxfqvE adAQpi119naNEOc24zn/tZAvOm++JqXSUymNnxV96y/k3esg9bWhzOiLbxRcuNBs5e odszu3eEZG+axxtfVKk9WsRbhNSkGYXY5+sARrQLydAe3LcYNsSrAFzJ4ITpn5qcE9 GJUHONG2kQL6Q== From: =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= To: Alexandre Ghiti , Albert Ou , David Hildenbrand , Palmer Dabbelt , Paul Walmsley , linux-riscv@lists.infradead.org, Oscar Salvador Cc: =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Andrew Bresticker , Chethan Seshadri , Lorenzo Stoakes , Santosh Mamila , Sivakumar Munnangi , Sunil V L , linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux-foundation.org Subject: [PATCH v3 0/9] riscv: Memory Hot(Un)Plug support Date: Tue, 21 May 2024 13:48:21 +0200 Message-Id: <20240521114830.841660-1-bjorn@kernel.org> X-Mailer: git-send-email 2.40.1 Precedence: bulk X-Mailing-List: virtualization@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Björn Töpel ================================================================ Memory Hot(Un)Plug support (and ZONE_DEVICE) for the RISC-V port ================================================================ Introduction ============ To quote "Documentation/admin-guide/mm/memory-hotplug.rst": "Memory hot(un)plug allows for increasing and decreasing the size of physical memory available to a machine at runtime." This series adds memory hot(un)plugging, and ZONE_DEVICE support for the RISC-V Linux port. MM configuration ================ RISC-V MM has the following configuration: * Memory blocks are 128M, analogous to x86-64. It uses PMD ("hugepage") vmemmaps. From that follows that 2M (PMD) worth of vmemmap spans 32768 pages á 4K which gets us 128M. * The pageblock size is the minimum minimum virtio_mem size, and on RISC-V it's 2M (2^9 * 4K). Implementation ============== The PGD table on RISC-V is shared/copied between for all processes. To avoid doing page table synchronization, the first patch (patch 1) pre-allocated the PGD entries for vmemmap/direct map. By doing that the init_mm PGD will be fixed at kernel init, and synchronization can be avoided all together. The following two patches (patch 2-3) does some preparations, followed by the actual MHP implementation (patch 4-5). Then, MHP and virtio-mem are enabled (patch 6-7), and finally ZONE_DEVICE support is added (patch 8). MHP and locking =============== TL;DR: The MHP does not step on any toes, except for ptdump. Additional locking is required for ptdump. Long version: For v2 I spent some time digging into init_mm synchronization/update. Here are my findings, and I'd love them to be corrected if incorrect. It's been a gnarly path... The `init_mm` structure is a special mm (perhaps not a "real" one). It's a "lazy context" that tracks kernel page table resources, e.g., the kernel page table (swapper_pg_dir), a kernel page_table_lock (more about the usage below), mmap_lock, and such. `init_mm` does not track/contain any VMAs. Having the `init_mm` is convenient, so that the regular kernel page table walk/modify functions can be used. Now, `init_mm` being special means that the locking for kernel page tables are special as well. On RISC-V the PGD (top-level page table structure), similar to x86, is shared (copied) with user processes. If the kernel PGD is modified, it has to be synched to user-mode processes PGDs. This is avoided by pre-populating the PGD, so it'll be fixed from boot. The in-kernel pgd regions are documented in `Documentation/arch/riscv/vm-layout.rst`. The distinct regions are: * vmemmap * vmalloc/ioremap space * direct mapping of all physical memory * kasan * modules, BPF * kernel Memory hotplug is the process of adding/removing memory to/from the kernel. Adding is done in two phases: 1. Add the memory to the kernel 2. Online memory, making it available to the page allocator. Step 1 is partially architecture dependent, and updates the init_mm page table: * Update the direct map page tables. The direct map is a linear map, representing all physical memory: `virt = phys + PAGE_OFFSET` * Add a `struct page` for each added page of memory. Update the vmemmap (virtual mapping to the `struct page`, so we can easily transform a kernel virtual address to a `struct page *` address. >From an MHP perspective, there are two regions of the PGD that are updated: * vmemmap * direct mapping of all physical memory The `struct mm_struct` has a couple of locks in play: * `spinlock_t page_table_lock` protects the page table, and some counters * `struct rw_semaphore mmap_lock` protect an mm's VMAs Note again that `init_mm` does not contain any VMAs, but still uses the mmap_lock in some places. The `page_table_lock` was originally used to to protect all pages tables, but more recently a split page table lock has been introduced. The split lock has a per-table lock for the PTE and PMD tables. If split lock is disabled, all tables are guarded by `mm->page_table_lock` (for user processes). Split page table locks are not used for init_mm. MHP operations is typically synchronized using `DEFINE_STATIC_PERCPU_RWSEM(mem_hotplug_lock)`. Actors ------ The following non-MHP actors in the kernel traverses (read), and/or modifies the kernel PGD. * `ptdump` Walks the entire `init_mm`, via `ptdump_walk_pgd()` with the `mmap_write_lock(init_mm)` taken. Observation: ptdump can race with MHP, and needs additional locking to avoid crashes/races. * `set_direct_*` / `arch/riscv/mm/pageattr.c` The `set_direct_*` functionality is used to "synchronize" the direct map to other kernel mappings, e.g. modules/kernel text. The direct map is using "as large huge table mappings as possible", which means that the `set_direct_*` might need to split the direct map. The `set_direct_*` functions operates with the `mmap_write_lock(init_mm)` taken. Observation: `set_direct_*` uses the direct map, but will never modify the same entry as MHP. If there is a mapping, that entry will never race with MHP. Further, MHP acts when memory is offline. * HVO / `mm/hugetlb_vmemmap` HVO optimizes the backing `struct page` for hugetlb pages, which means changing the "vmemmap" region. HVO can split (merge?) a vmemmap pmd. However, it will never race with MHP, since HVO only operates at online memory. HVO cannot touch memory being MHP added or removed. * `apply_to_page_range` Walks a range, creates pages and applies a callback (setting permissions) for the page. When creating a table, it might use `int __pte_alloc_kernel(pmd_t *pmd)` which takes the `init_mm.page_table_lock` to synchronize pmd populate. Used by: `mm/vmalloc.c` and `mm/kasan/shadow.c`. The KASAN callback takes the `init_mm.page_table_lock` to synchronize pte creation. Observations: `apply_to_page_range` applies to the "vmalloc/ioremap space" region, and "kasan" region. *Not* affected by MHP. * `apply_to_existing_page_range` Walks a range, applies a callback (setting permissions) for the page (no page creation). Used by: `kernel/bpf/arena.c` and `mm/kasan/shadow.c`. The KASAN callback takes the `init_mm.page_table_lock` to synchronize pte creation. *Not* affected by MHP regions. * `apply_to_existing_page_range` applies to the "vmalloc/ioremap space" region, and "kasan" region. *Not* affected by MHP regions. * `ioremap_page_range` and `vmap_page_range` Uses the same internal function, and might create table entries at the "vmalloc/ioremap space" region. Can call `__pte_alloc_kernel()` which takes the `init_mm.page_table_lock` synchronizing pmd populate in the region. *Not* affected by MHP regions. Summary: * MHP add will never modify the same page table entries, as any of the other actors. * MHP remove is done when memory is offlined, and will not clash with any of the actors. * Functions that walk the entire kernel page table need synchronization * It's sufficient to add the MHP lock ptdump. Testing ======= This series adds basic DT supported hotplugging. There is a QEMU series enabling MHP for the RISC-V "virt" machine here: [1] ACPI/MSI support is still in the making for RISC-V, and prior proper (ACPI) PCI MSI support lands [2] and NUMA SRAT support [3], it hard to try it out. I've prepared a QEMU branch with proper ACPI GED/PC-DIMM support [4], and a this series with the required prerequisites [5] (AIA, ACPI AIA MADT, ACPI NUMA SRAT). To test with virtio-mem, e.g.: | qemu-system-riscv64 \ | -machine virt,aia=aplic-imsic \ | -cpu rv64,v=true,vlen=256,elen=64,h=true,zbkb=on,zbkc=on,zbkx=on,zkr=on,zkt=on,svinval=on,svnapot=on,svpbmt=on \ | -nodefaults \ | -nographic -smp 8 -kernel rv64-u-boot.bin \ | -drive file=rootfs.img,format=raw,if=virtio \ | -device virtio-rng-pci \ | -m 16G,slots=3,maxmem=32G \ | -object memory-backend-ram,id=mem0,size=16G \ | -numa node,nodeid=0,memdev=mem0 \ | -serial chardev:char0 \ | -mon chardev=char0,mode=readline \ | -chardev stdio,mux=on,id=char0 \ | -device pci-serial,id=serial0,chardev=char0 \ | -object memory-backend-ram,id=vmem0,size=2G \ | -device virtio-mem-pci,id=vm0,memdev=vmem0,node=0 where "rv64-u-boot.bin" is U-boot with EFI/ACPI-support (use [6] if you're lazy). In the QEMU monitor: | (qemu) info memory-devices | (qemu) qom-set vm0 requested-size 1G ...to test DAX/KMEM, use the follow QEMU parameters: | -object memory-backend-file,id=mem1,share=on,mem-path=virtio_pmem.img,size=4G \ | -device virtio-pmem-pci,memdev=mem1,id=nv1 and the regular ndctl/daxctl dance. If you're brave to try the ACPI branch, add "acpi=on" to "-machine virt", and test PC-DIMM MHP (in addition to virtio-{p},mem): In the QEMU monitor: | (qemu) object_add memory-backend-ram,id=mem1,size=1G | (qemu) device_add pc-dimm,id=dimm1,memdev=mem1 You can also try hot-remove with some QEMU options, say: | -object memory-backend-file,id=mem-1,size=256M,mem-path=/pagesize-2MB | -device pc-dimm,id=mem1,memdev=mem-1 | -object memory-backend-file,id=mem-2,size=1G,mem-path=/pagesize-1GB | -device pc-dimm,id=mem2,memdev=mem-2 | -object memory-backend-file,id=mem-3,size=256M,mem-path=/pagesize-2MB | -device pc-dimm,id=mem3,memdev=mem-3 Remove "acpi=on" to run with DT. Thanks to Alex, Andrew, David, and Oscar for all comments/tests/fixups. Changelog ========= v2->v3: Collect tags 0001-riscv-mm-Properly-forward-vmemmap_populate-altmap * New patch, split from attribute change (Alex, Oscar) 0005-riscv-mm-Add-memory-hotplugging-support * Fix page table free to take PG_reserved into consideration (David, Oscar) * Depend on SPARSE_VMEMMAP (Alex, David, Oscar) * Proper TLB flushing (Alex) 0009-riscv-mm-Add-support-for-ZONE_DEVICE * Fixed 32b build v1->v2: Reviewed a lot of MHP locking scenarios Various config build issues (e.g. build !CONFIG_MEMORY_HOTPLUG) (Andrew) Added arch_get_mappable_range() implementation Acquire MHP lock for ptdump, analogous to arm64 ACPI MHP tests Add ZONE_DEVICE patch References ========== [1] https://lore.kernel.org/qemu-devel/20240521105635.795211-1-bjorn@kernel.org/ [2] https://lore.kernel.org/linux-riscv/20240501121742.1215792-1-sunilvl@ventanamicro.com/ [3] https://lore.kernel.org/linux-riscv/cover.1713778236.git.haibo1.xu@intel.com/ [4] https://github.com/bjoto/qemu/commits/virtio-mem-pc-dimm-mhp-acpi-v2/ [5] https://github.com/bjoto/linux/commits/mhp-v3-next-acpi [6] https://github.com/bjoto/riscv-rootfs-utils/tree/acpi Thanks, Björn Björn Töpel (9): riscv: mm: Properly forward vmemmap_populate() altmap parameter riscv: mm: Pre-allocate vmemmap/direct map PGD entries riscv: mm: Change attribute from __init to __meminit for page functions riscv: mm: Refactor create_linear_mapping_range() for memory hot add riscv: mm: Add memory hotplugging support riscv: mm: Take memory hotplug read-lock during kernel page table dump riscv: Enable memory hotplugging for RISC-V virtio-mem: Enable virtio-mem for RISC-V riscv: mm: Add support for ZONE_DEVICE arch/riscv/Kconfig | 3 + arch/riscv/include/asm/kasan.h | 4 +- arch/riscv/include/asm/mmu.h | 4 +- arch/riscv/include/asm/pgtable-64.h | 20 ++ arch/riscv/include/asm/pgtable-bits.h | 1 + arch/riscv/include/asm/pgtable.h | 19 +- arch/riscv/mm/init.c | 339 +++++++++++++++++++++++--- arch/riscv/mm/ptdump.c | 3 + drivers/virtio/Kconfig | 2 +- 9 files changed, 349 insertions(+), 46 deletions(-) base-commit: 92cce91949a497a8a4615f9ba5813b03f7a1f1d5 -- 2.40.1