From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E2B3C4742C for ; Sat, 14 Nov 2020 02:43:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 41C312222F for ; Sat, 14 Nov 2020 02:43:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="H6WD2qDd"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mg.codeaurora.org header.i=@mg.codeaurora.org header.b="X661P4za" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 41C312222F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-ID:Subject:To:From:Date:MIME-Version: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=diqDnf144Ku0ZPULkllRFbima9HcXwNfbQGwCOks+BE=; b=H6WD2qDdN1IpyP hypT3hKO0CFxrCXZjtcqiJd6AeBzaFVcNe2zWX1W6ukYGHvB+yqurIhSrlUCTsPn4oA875q4FUwxC Fz5dc51ooHaEjfFi6D7Y8WoSTw0sp7X7IdTQrxv3VmpjiPpnYTVoWLkpaGFWWiQOiHPX0IniH/r6o 5fDkXo58EUIZ0K7mzHlkP+xImHN72HCiYdou3lS5GxJ6slROASi56MpAGfXBK35JPhJ8ZJr4SpFVB tlcUidyb/EJOn5v2CdRQSIwGrmYThW6EVc2Ebd+ePcgqFAwurBZ2OFMCz3dF1hHIg2Hb0rhgWwtR5 9PkQ5GyFhj3sljnQt4ZQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdlVJ-0004mh-3G; Sat, 14 Nov 2020 02:41:29 +0000 Received: from m42-4.mailgun.net ([69.72.42.4]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdlVG-0004mK-63 for linux-arm-kernel@lists.infradead.org; Sat, 14 Nov 2020 02:41:27 +0000 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.codeaurora.org; q=dns/txt; s=smtp; t=1605321686; h=Message-ID: Subject: To: From: Date: Content-Transfer-Encoding: Content-Type: MIME-Version: Sender; bh=EB58wdI9Dt3Cl+zcCD+PGvSh+hKiJNn9Kj37xFgOX3w=; b=X661P4za0xUhRHTXRCrMQQMz6coLdP8zuZ14puVVjHGMJ26iO7+qJv3k9RFCPtSkew4PXNqV D3Q3CrwJdUqWnd1+kMX4EC9KVnlmZhfeoklXSzcI2mRD7MSA6x9T1rl8KlnHN9JjPXkJCTwI 5kcwtfR/zi2EaI/3YE/eAiJyeh0= X-Mailgun-Sending-Ip: 69.72.42.4 X-Mailgun-Sid: WyJiYzAxZiIsICJsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmciLCAiYmU5ZTRhIl0= Received: from smtp.codeaurora.org (ec2-35-166-182-171.us-west-2.compute.amazonaws.com [35.166.182.171]) by smtp-out-n02.prod.us-east-1.postgun.com with SMTP id 5faf43d4c3c3b09004eb2284 (version=TLS1.2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256); Sat, 14 Nov 2020 02:41:24 GMT Received: by smtp.codeaurora.org (Postfix, from userid 1001) id 6B4B9C433C8; Sat, 14 Nov 2020 02:41:23 +0000 (UTC) Received: from mail.codeaurora.org (localhost.localdomain [127.0.0.1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sudaraja) by smtp.codeaurora.org (Postfix) with ESMTPSA id A07A6C433C6; Sat, 14 Nov 2020 02:41:22 +0000 (UTC) MIME-Version: 1.0 Date: Fri, 13 Nov 2020 18:41:22 -0800 From: Sudarshan Rajagopalan To: Anshuman Khandual , David Hildenbrand , Mark Rutland , Steven Price , Will Deacon , Catalin Marinas , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Suren Baghdasaryan , Pratik Patel Subject: [RFC] depopulate_range_driver_managed() for removing page-table mappings for hot-added memory blocks Message-ID: X-Sender: sudaraja@codeaurora.org User-Agent: Roundcube Webmail/1.3.9 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201113_214126_519098_5F1A98EE X-CRM114-Status: GOOD ( 12.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org CkhlbGxvLAoKV2hlbiBtZW1vcnkgYmxvY2tzIGFyZSByZW1vdmVkLCBhbG9uZyB3aXRoIHJlbW92 aW5nIHRoZSBtZW1tYXAgZW50cmllcywgCm1lbW9yeSByZXNvdXJjZSBhbmQgbWVtb3J5IGJsb2Nr IGRldmljZXMsIHRoZSBhcmNoIHNwZWNpZmljIAphcmNoX3JlbW92ZV9tZW1vcnkoKSBpcyBjYWxs ZWQgd2hpY2ggdGFrZXMgY2FyZSBvZiB0ZWFyaW5nIGRvd24gdGhlIApwYWdlLXRhYmxlcy4KClN1 cHBvc2UgdGhlcmXigJlzIGEgdXNlY2FzZSB3aGVyZSB0aGUgcmVtb3ZlZCBtZW1vcnkgYmxvY2tz IHdpbGwgYmUgYWRkZWQgCmJhY2sgaW50byB0aGUgc3lzdGVtIGF0IGxhdGVyIHBvaW50LCB3ZSBj YW4gcmVtb3ZlL29mZmxpbmUgdGhlIGJsb2NrIGluIAphIHdheSB0aGF0IGFsbCBlbnRyaWVzIHN1 Y2ggYXMgbWVtbWFwcywgbWVtb3J5IHJlc291cmNlcyBhbmQgYmxvY2sgCmRldmljZXMgY2FuIGJl IGtlcHQgaW50YWN0IHNvIHRoYXQgdGhleSB3b27igJl0IGJlIG5lZWRlZCB0byBiZSBjcmVhdGVk IAphZ2FpbiB3aGVuIGJsb2NrcyBhcmUgYWRkZWQgYmFjay4gTm93IHRoaXMgY2FuIGJlIGRvbmUg YnkgZG9pbmcgb2ZmbGluZSAKYWxvbmUuIEJ1dCBpZiB0aGVyZeKAmXMgc3BlY2lhbCB1c2VjYXNl IHdoZXJlIHRoZSBwYWdlLXRhYmxlIGVudHJpZXMgYXJlIApuZWVkZWQgdG8gYmUgdGVhcmVkIGRv d24gd2hlbiBibG9ja3MgYXJlIG9mZmxpbmVkIGluIG9yZGVyIHRvIGF2b2lkIApzcGVjdWxhdGl2 ZSBhY2Nlc3NlcyBvbiBvZmZsaW5lZCBtZW1vcnkgcmVnaW9uLCBidXQgYWxzbyBrZWVwIHRoZSBt ZW1tYXAgCmVudHJpZXMgYW5kIGJsb2NrIGRldmljZXMgaW50YWN0LCBJIHdhcyB0aGlua2luZyBp ZiB3ZSBjYW4gaW1wbGVtZW50IApzb21ldGhpbmcgbGlrZSB7cG9wdWxhdGV8ZGVwb3B1bGF0ZX1f cmFuZ2VfZHJpdmVyX21hbmFnZWQoKSB0aGF0IGNhbiBiZSAKY2FsbGVkIGFmdGVyIG9ubGluZS9v ZmZsaW5lIHdoaWNoIGNhbiBjcmVhdGUvdGVhciBkb3duIHBhZ2UgdGFibGUgCm1hcHBpbmdzIGZv ciB0aGF0IHJhbmdlLiBUaGlzIHdvdWxkIGF2b2lkIHVzIGZyb20gdGhlIG5lZWQgdG8gZG8gCnJl bW92ZV9tZW1vcnkoKSBlbnRpcmVseSBqdXN0IGZvciB0aGUgc2FrZSBvZiBwYWdlLXRhYmxlIGVu dHJpZXMgYmVpbmcgCnJlbW92ZWQuIFdlIGNhbiBub3cganVzdCBvZmZsaW5lIHRoZSBibG9jayBh bmQgY2FsbCAKZGVwb3B1bGF0ZV9yYW5nZV9kcml2ZXJfbWFuYWdlZC4KClRoaXMgYmFzaWNhbGx5 IGlzb2xhdGVzIGFyY2hfe2FkZC9yZW1vdmV9X21lbW9yeSBvdXRzaWRlIG9mIAphZGQvcmVtb3Zl X21lbW9yeSByb3V0aW5lcyBzbyB0aGF0IGRyaXZlcnMgY2FuIGNob29zZSBpZiBpdCBuZWVkcyB0 byAKanVzdCBvZmZsaW5lIGFuZCByZW1vdmUgcGFnZS10YWJsZSBtYXBwaW5ncyBvciBob3RyZW1v dmUgbWVtb3J5IAplbnRpcmVseS4gVGhpcyBnaXZlcyBkcml2ZXJzIHRoZSBmbGV4aWJpbGl0eSB0 byByZXRhaW4gbWVtbWFwIGVudHJpZXMgCmFuZCBtZW1vcnkgcmVzb3VyY2UgYW5kIGJsb2NrIGRl dmljZSBjcmVhdGlvbiBzbyB0aGF0IHRoZXkgY2FuIGJlIApza2lwcGVkIHdoZW4gYmxvY2tzIGFy ZSBhZGRlZCBiYWNrIOKAkyB0aGlzIGhlbHBzIHVzIHJlZHVjZSB0aGUgbGF0ZW5jaWVzIApmb3Ig cmVtb3ZpbmcgYW5kIGFkZGluZyBtZW1vcnkgYmxvY2tzLgoKSeKAmW0gc3RpbGwgaW4gdGhlIHBy b2Nlc3MgdGhlIGNyZWF0aW5nIHRoZSBwYXRjaCB0aGF0IGltcGxlbWVudHMgdGhpcywgCndoaWNo IHdvdWxkIGdpdmUgY2xlYXIgdmlldyBhYm91dCB0aGlzIFJGQyBidXQganVzdCBwdXR0aW5nIG91 dCB0aGUgCnRob3VnaHQgaGVyZSBpZiBpdCBtYWtlcyBzZW5zZSBvciBub3QuCgoKU3VkYXJzaGFu Ci0tClF1YWxjb21tIElubm92YXRpb24gQ2VudGVyLCBJbmMuIGlzIGEgbWVtYmVyIG9mIENvZGUg QXVyb3JhIEZvcnVtLCBhIApMaW51eCBGb3VuZGF0aW9uIENvbGxhYm9yYXRpdmUgUHJvamVjdAoK X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJt LWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3Jn Cmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtl cm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77546C4742C for ; Sat, 14 Nov 2020 02:41:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2456222240 for ; Sat, 14 Nov 2020 02:41:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=mg.codeaurora.org header.i=@mg.codeaurora.org header.b="SvXNtrSa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726288AbgKNClc (ORCPT ); Fri, 13 Nov 2020 21:41:32 -0500 Received: from z5.mailgun.us ([104.130.96.5]:46264 "EHLO z5.mailgun.us" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726116AbgKNClb (ORCPT ); Fri, 13 Nov 2020 21:41:31 -0500 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.codeaurora.org; q=dns/txt; s=smtp; t=1605321691; h=Message-ID: Subject: To: From: Date: Content-Transfer-Encoding: Content-Type: MIME-Version: Sender; bh=EB58wdI9Dt3Cl+zcCD+PGvSh+hKiJNn9Kj37xFgOX3w=; b=SvXNtrSaR8MH5CITv0GjLfSCLtLXh/+6bq1V5wmyk4yq38GnZG4RApcSqtn1z37PHX/P3Wmw PaljZO5B+YYcHJpCQgqjbczBvjB47yLSB8VxA4JFslyTGJiee6WH+xyDqWUQ5WEtWkHJJBSc y2ytMMoz7/2PcY40UYG5XfGeXdU= X-Mailgun-Sending-Ip: 104.130.96.5 X-Mailgun-Sid: WyI0MWYwYSIsICJsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnIiwgImJlOWU0YSJd Received: from smtp.codeaurora.org (ec2-35-166-182-171.us-west-2.compute.amazonaws.com [35.166.182.171]) by smtp-out-n01.prod.us-west-2.postgun.com with SMTP id 5faf43d324ba9b3b02af8094 (version=TLS1.2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256); Sat, 14 Nov 2020 02:41:23 GMT Sender: sudaraja=codeaurora.org@mg.codeaurora.org Received: by smtp.codeaurora.org (Postfix, from userid 1001) id 72CACC433FF; Sat, 14 Nov 2020 02:41:23 +0000 (UTC) Received: from mail.codeaurora.org (localhost.localdomain [127.0.0.1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sudaraja) by smtp.codeaurora.org (Postfix) with ESMTPSA id A07A6C433C6; Sat, 14 Nov 2020 02:41:22 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Fri, 13 Nov 2020 18:41:22 -0800 From: Sudarshan Rajagopalan To: Anshuman Khandual , David Hildenbrand , Mark Rutland , Steven Price , Will Deacon , Catalin Marinas , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Suren Baghdasaryan , Pratik Patel Subject: [RFC] depopulate_range_driver_managed() for removing page-table mappings for hot-added memory blocks Message-ID: X-Sender: sudaraja@codeaurora.org User-Agent: Roundcube Webmail/1.3.9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, When memory blocks are removed, along with removing the memmap entries, memory resource and memory block devices, the arch specific arch_remove_memory() is called which takes care of tearing down the page-tables. Suppose there’s a usecase where the removed memory blocks will be added back into the system at later point, we can remove/offline the block in a way that all entries such as memmaps, memory resources and block devices can be kept intact so that they won’t be needed to be created again when blocks are added back. Now this can be done by doing offline alone. But if there’s special usecase where the page-table entries are needed to be teared down when blocks are offlined in order to avoid speculative accesses on offlined memory region, but also keep the memmap entries and block devices intact, I was thinking if we can implement something like {populate|depopulate}_range_driver_managed() that can be called after online/offline which can create/tear down page table mappings for that range. This would avoid us from the need to do remove_memory() entirely just for the sake of page-table entries being removed. We can now just offline the block and call depopulate_range_driver_managed. This basically isolates arch_{add/remove}_memory outside of add/remove_memory routines so that drivers can choose if it needs to just offline and remove page-table mappings or hotremove memory entirely. This gives drivers the flexibility to retain memmap entries and memory resource and block device creation so that they can be skipped when blocks are added back – this helps us reduce the latencies for removing and adding memory blocks. I’m still in the process the creating the patch that implements this, which would give clear view about this RFC but just putting out the thought here if it makes sense or not. Sudarshan -- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project