From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7DB5C41513 for ; Wed, 2 Aug 2023 15:57:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 567462801C2; Wed, 2 Aug 2023 11:57:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 518C52801AA; Wed, 2 Aug 2023 11:57:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B9B92801C2; Wed, 2 Aug 2023 11:57:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2C3A62801AA for ; Wed, 2 Aug 2023 11:57:17 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EA2C61A0F4B for ; Wed, 2 Aug 2023 15:57:16 +0000 (UTC) X-FDA: 81079618872.10.2E61346 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf25.hostedemail.com (Postfix) with ESMTP id 5F802A000D for ; Wed, 2 Aug 2023 15:57:14 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=rNzKRaea; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf25.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690991834; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UsmY3/6Kn8L9keeIZr04vLXviQCHfw9UeyBPatlFKjs=; b=YrpzEYAZzlxWM40FLtbxZVapM2dQysKR3ip2ROG5Va3x+vm8d/9zPgnEm/hO29Mcy7Ckv3 BkD0DX3PsDWsNBB1hbTl8N2eF2bxM337t6PZnpeQjC2SM5hqrrWAkqrQBOL1IWzoMmcOI9 1H9jA6SDMExLLtKSrs7wqPuO8gd8ePM= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=rNzKRaea; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf25.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690991834; a=rsa-sha256; cv=none; b=0XP2JVqIaAD3FXqYkhep+HFISg+G6Pg5D1gK7pp1l3su8tc+Jdc3sCw6CnWHLtqnYrWK5K ewnUGo7zAjymnOv+tgBTi2mCL7lb+go2LsxAK5NMTJDXo1Wmch+K5I0fEtC0qJJOBMEeFD fRZXWI9RYLjLXU0OrS8YF9qfNgzIRGs= Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 372FmfMj016758; Wed, 2 Aug 2023 15:57:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pp1; bh=UsmY3/6Kn8L9keeIZr04vLXviQCHfw9UeyBPatlFKjs=; b=rNzKRaeaDUhCTPYW1r47bItWLsHSXwuSCYirhbYzh3tUGEfpA96V1E7O50ygjtr1vNFE PP0D30i8BcrNcPc/Q2vXN2vO2M50/NL5cdBxmMWjzp02Mczk59oxkdGicQ1AHook8FWI k4IG8x3uPjj3TLpskFk3xLr4qv5uFd/lksyoVYVM9mnkiVFDwr9EMyRoIIPlQh7jvq/i 96xzjTxWurKugs+U0qh24NQr+QASVWZcbC+4O/NzV+6sHK18BsLJFuFrRwHzTZy6O8/+ 9HhCQE9Yl6WIha5UmX9xebbBC05E3Cd/VxJrEPYjUXrNSQB5bbOfucf+Bkky1wpPs10S sg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3s7t45r5jn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Aug 2023 15:57:06 +0000 Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 372FmjCT016927; Wed, 2 Aug 2023 15:57:06 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3s7t45r5j5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Aug 2023 15:57:06 +0000 Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 372FeOmO014537; Wed, 2 Aug 2023 15:57:05 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 3s5ft1n698-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Aug 2023 15:57:04 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 372Fv3sK38142626 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 2 Aug 2023 15:57:03 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5216E20043; Wed, 2 Aug 2023 15:57:03 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5942E20040; Wed, 2 Aug 2023 15:57:01 +0000 (GMT) Received: from [9.43.11.83] (unknown [9.43.11.83]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 2 Aug 2023 15:57:01 +0000 (GMT) Message-ID: <6f66dfcc-3d2a-c47c-1b84-c0d5210564d6@linux.ibm.com> Date: Wed, 2 Aug 2023 21:27:00 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v7 7/7] mm/memory_hotplug: Enable runtime update of memmap_on_memory parameter Content-Language: en-US To: David Hildenbrand , Michal Hocko Cc: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu, Oscar Salvador , Vishal Verma References: <20230801044116.10674-1-aneesh.kumar@linux.ibm.com> <20230801044116.10674-8-aneesh.kumar@linux.ibm.com> <31305ab7-1e65-80aa-ee91-9190c8f67430@redhat.com> From: Aneesh Kumar K V In-Reply-To: <31305ab7-1e65-80aa-ee91-9190c8f67430@redhat.com> Content-Type: text/plain; charset=UTF-8 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: xyuVu8gHNdrGGVc6nLYPzEg4Qgmf2wli X-Proofpoint-ORIG-GUID: _XITVjbX84zCYtpnyH8n390p-lCvpnvN Content-Transfer-Encoding: 7bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-08-02_12,2023-08-01_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxlogscore=842 suspectscore=0 priorityscore=1501 mlxscore=0 spamscore=0 impostorscore=0 phishscore=0 malwarescore=0 adultscore=0 bulkscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2308020138 X-Rspamd-Queue-Id: 5F802A000D X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 58idwkc8otj4nxqumqh5w6tmdr45c18k X-HE-Tag: 1690991834-803347 X-HE-Meta: U2FsdGVkX1+d4fcEKz/F+PuXXZ++MB8B/tbEEWBDz9u8gPMRLMhcsglIcSYikH5N1SF82sA9Elxjb22/jQaJFUekxwnE5nKetD324Mcv5uTyybSfMcQiVBH5BAdvE29JRYgpfwxTZ183xvW1IZyIySeaum7hk6PDvWWMMYettVnXGnGeB9xfFtlznXbcSy174XG1baopZNTLPdaWPm1wju2YPSCpVVTmX4LxrwUORiAHvweYsOEUbHUIXdH+S17nmF91haOPO9MIfbr4gkAsVli/IhQljFUyOAZkC7B/kfuz23vwoRxKTZccx3PqTNXlQW6UktoIIi+71U0UQjNAbUQjs4ZsZccZwx4LxzwR9iv6kPlM+uq8xLNdRARbGLKG9TSTnN8IugF3/EJqkrduFO/m4k2STPmM4eu69c9y78MJ1R60ogbvZqZCMuQaui1YY3oKOPW/pyxudTYYV7xj/kEd39Cfcp3yAizQwT+R0eWZ4LsylJ27I2QUmKo6cCKLWLwXiaI1PzBydho8mFvIKQbryxUvKWE0ZHGMmpbflzXd4tGoy2F7GiVPaoNp+G42Zfc7iHvkTDSpTSMeqv+If0EWy9a3iAe/1e8PvDeg4ogZaalRfPLG/zCTx4wJE6Atm0up39+kK94ECkmOmqE0Tmfx2G6VoJpwYCBb22SV0YULF8+JkI1QtuoU3qNGUONT8V6xmhx9Z87IEErt7OkgMWgFavPBokspCCNtTKDJ98KQe+HtQMlLH9XKUjfgkAY9+oBF3PuAU/QpGRUAx5hiM4isg6dxhLlG9CYTsdMDC1C8Fd79yZxCNRI8vj/niKXmkn8QdUkbWWhSc0uwXiahL2yQ1FRUdAt/J9ynHe+qub8xbAGU3qAI1SuP+lhvi1Zx4PeJGdO1MDjdaDYf++NxPhabBuDJHoc3ly7sR4yh3178cs1docTehOSKDIbe8ZMA+Jgp/s7Qk9VsTE41Trc mIFj8adx +dhmTZFs2KRkQMd/kYG6qQgjWn+cC+UujbUo6HFyOXKhOid0KT1buI3g5nBV8IY4fViKJvWhsGoHAEu8RfxU7i5z4jzOohw5aYGD8UtBeyYer8HyFjGasTArh5Nz1mDjATDFk9KCsI/mQ0EzKqUs0dGflCa7RtOrhvmLxoonZLWRlramcuBHlLMLMLT1Ge3/C+xvHME7WoygXh7vixMxeOOKhIQwkL7vMUijk7d9t89scSi+F03U6eP/kv8PsVNMsHrVxidFH15EN/GogB//0diTCzkKdB0RuvVzumnZwepzPl+iu2O8x0zGHxQ0+7axs/S+0l6RjhX4PH+m9KnHR6VzhJBr8IakRbvc6hbVvHkO7SAGVz7NH4MLBw9coFXZXhaRf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 8/2/23 9:24 PM, David Hildenbrand wrote: > On 02.08.23 17:50, Michal Hocko wrote: >> On Wed 02-08-23 10:15:04, Aneesh Kumar K V wrote: >>> On 8/1/23 4:20 PM, Michal Hocko wrote: >>>> On Tue 01-08-23 14:58:29, Aneesh Kumar K V wrote: >>>>> On 8/1/23 2:28 PM, Michal Hocko wrote: >>>>>> On Tue 01-08-23 10:11:16, Aneesh Kumar K.V wrote: >>>>>>> Allow updating memmap_on_memory mode after the kernel boot. Memory >>>>>>> hotplug done after the mode update will use the new mmemap_on_memory >>>>>>> value. >>>>>> >>>>>> Well, this is a user space kABI extension and as such you should spend >>>>>> more words about the usecase. Why we could live with this static and now >>>>>> need dynamic? >>>>>> >>>>> >>>>> This enables easy testing of memmap_on_memory feature without a kernel reboot. >>>> >>>> Testing alone is rather weak argument to be honest. >>>> >>>>> I also expect people wanting to use that when they find dax kmem memory online >>>>> failing because of struct page allocation failures[1]. User could reboot back with >>>>> memmap_on_memory=y kernel parameter. But being able to enable it via sysfs makes >>>>> the feature much more useful. >>>> >>>> Sure it can be useful but that holds for any feature, right. The main >>>> question is whether this is worth maintaing. The current implementation >>>> seems rather trivial which is an argument to have it but are there any >>>> risks long term? Have you evaluated a potential long term maintenance >>>> cost? There is no easy way to go back and disable it later on without >>>> breaking some userspace. >>>> >>>> All that should be in the changelog! >>> >>> I updated it as below. >>> >>> mm/memory_hotplug: Enable runtime update of memmap_on_memory parameter >>> >>> Allow updating memmap_on_memory mode after the kernel boot. Memory >>> hotplug done after the mode update will use the new mmemap_on_memory >>> value. >>> >>> It is now possible to test the memmap_on_memory feature easily without >>> the need for a kernel reboot. Additionally, for those encountering >>> struct page allocation failures while using dax kmem memory online, this >>> feature may prove useful. Instead of rebooting with the >>> memmap_on_memory=y kernel parameter, users can now enable it via sysfs, >>> which greatly enhances its usefulness. >> >> >> I do not really see a solid argument why rebooting is really a problem >> TBH. Also is the global policy knob really the right fit for existing >> hotplug usecases? In other words, if we really want to make >> memmap_on_memory more flexible would it make more sense to have it per >> memory block property instead (the global knob being the default if >> admin doesn't specify it differently). > > Per memory block isn't possible, due to the creation order. Also, I think it's not the right approach. > > I thought about driver toggles. At least for dax/kmem people are looking into that: > > https://lkml.kernel.org/r/20230801-vv-kmem_memmap-v3-2-406e9aaf5689@intel.com > > Where that can also be toggled per device. > That still is dependent on the global memmap_on_memory enabled right? We could make the dax facility independent of the global feature toggle? -aneesh