From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05304C001B0 for ; Tue, 8 Aug 2023 06:29:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 325F76B0071; Tue, 8 Aug 2023 02:29:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D6B86B0074; Tue, 8 Aug 2023 02:29:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1773B8D0001; Tue, 8 Aug 2023 02:29:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 080516B0071 for ; Tue, 8 Aug 2023 02:29:51 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CC1DB14073F for ; Tue, 8 Aug 2023 06:29:50 +0000 (UTC) X-FDA: 81099961740.13.DC9EFE6 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf20.hostedemail.com (Postfix) with ESMTP id 7E2391C001C for ; Tue, 8 Aug 2023 06:29:48 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Wnd32b7W; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf20.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691476188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/K+iyWmpHfwFJUA4J+GYS5VEGZyeglN1XkoMxkXLjpQ=; b=ICsYTLqM/QOPbo4KwA6FjF1Nus2VPNAx98wTZmFSn18EZd+ZbCx6pd9Lx9Rv9obTn6tmv8 FKYKjByhCJdsQ/qsAoFUWIiNZQKqU5J4lnKs6n2rh4QHB0D3ilKu9QSbdhNHXRRIodAuo5 VGz5jpOHxRERWZSfY+ZF5bpE5OgByI8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Wnd32b7W; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf20.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691476188; a=rsa-sha256; cv=none; b=z5jq9pWMcfAVGYXQK1cnqmKAamKoFVIkBJ/VFUzly4W9ZGTlSgp9HY0ZVP03sdE/jLgIIx dq4KDDVt736RrEGbKPBq4XEHF9+4Fc3DxfCedytFMXLibUp/M/suBzS8ehFUsTGq61CBEt jKLK/uL2TubLm+BDMGfZ5QAZ9DOrzS0= Received: from pps.filterd (m0353723.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3786LFlh001796; Tue, 8 Aug 2023 06:29:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=/K+iyWmpHfwFJUA4J+GYS5VEGZyeglN1XkoMxkXLjpQ=; b=Wnd32b7WZ3n8HpMJLUzzF2UyYrUh9mqdicgHHdKMISLegQdK/dhBjzhlI4wmKtkVr8K1 Zbzmb1aHsKgpwuJr9yLAnVHpSBwa+GOgqK5s09RDNnP5ytk8un5nKYchd6WUhLSMqQRl aaUufdC4dYcgRnzCgSHzPtcbCG5Fszrir1dopT2oDlV2m6FjAUwLimjsL2q97kvd3OcA Q83VKJunkPNm4BdT8d3laZl8K9dbm5YGK32ly1E8RtgiUVC68FP9wuRrmpvT6E+86yoh jcTERUXgY0fcvnbI5/mCN9Zl7tiZeN7cznczwl5Yn3Wz3rPa/YN+CllJPs3GubYuZFV0 Tg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3sbg4wrhha-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 08 Aug 2023 06:29:39 +0000 Received: from m0353723.ppops.net (m0353723.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3786NhAH009057; Tue, 8 Aug 2023 06:29:39 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3sbg4wrhg8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 08 Aug 2023 06:29:39 +0000 Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 37864YBJ001772; Tue, 8 Aug 2023 06:29:37 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 3sa3f1khrg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 08 Aug 2023 06:29:37 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 3786TaqA39846272 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 8 Aug 2023 06:29:36 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 222792004B; Tue, 8 Aug 2023 06:29:36 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4B04720040; Tue, 8 Aug 2023 06:29:34 +0000 (GMT) Received: from [9.109.212.144] (unknown [9.109.212.144]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 8 Aug 2023 06:29:34 +0000 (GMT) Message-ID: <41ffec19-775a-534b-b217-438d3bd8b94e@linux.ibm.com> Date: Tue, 8 Aug 2023 11:59:33 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v7 7/7] mm/memory_hotplug: Enable runtime update of memmap_on_memory parameter Content-Language: en-US To: David Hildenbrand , Michal Hocko Cc: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu, Oscar Salvador , Vishal Verma References: <31305ab7-1e65-80aa-ee91-9190c8f67430@redhat.com> <1c6a74f0-85e9-5299-1520-9068e842b1a5@redhat.com> From: Aneesh Kumar K V In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: jcR45jUlNkJaaYiS7nzigDjHGol2Jlxe X-Proofpoint-GUID: GmlyqABiyau630WTndx2icKHqAnytggP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-08-08_04,2023-08-03_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 impostorscore=0 spamscore=0 phishscore=0 priorityscore=1501 lowpriorityscore=0 mlxlogscore=810 mlxscore=0 adultscore=0 clxscore=1011 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2308080054 X-Rspam-User: X-Stat-Signature: 87iakeqxbf9hcy8xb1x4p4fey7qt8ysu X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 7E2391C001C X-HE-Tag: 1691476188-280672 X-HE-Meta: U2FsdGVkX19fchUv62jgBBOc3I221eAMcLVtMxts/L96QTktcTc3TYKqJ50Ldxzu+Ck5EkkVvefA5i3CA/w92FJOlCBnufxPLU/W+q5q8bnagQQaSn8XuWlyh8ANoalLVwViYtEwnWTTa8RGjb3u642us2XLxFjTvXtFomCpbsvJ+r/FBowxo4G8u/QTfhmqcuJ4qV3iulC73OenP/eZIADBYLK8aPR4V2p/4qGCsAjMTcMCr1ADmrtZnAIzqjG6EsXQyloK7B4ko9nU9jdPnf/5XC7czxv+1LvIdNyxvf5ij+dX4HFtXofoO5stv0XhJNJ3bmZLb5ZfzlaghCVCok1zvpOiH6l/6WJoYK3I7rOYo4YZMuntVlRq2amRo6YsIln+oJlaS8aP5g20ha8X13QM9wNiUb2SJsDSsk8RXmTVa7Z5ufoc1amWrh5Gm2V9NL1rwD/WA8FgVGZmI+R8Dqcfz01qS1gAFsN30WGx9LUlfamyWYjpC/cd9Z5I0MovZ8ZsfCMT09QvqilQY1LBD1lo2nrXKjD1NFSF9L8IDYFupKBmIbsmCaLfqs3QVOm7h3WE2WMLswTNUqGXJt0ZxbKtMNxNmOLYVbNuvT+UMAdxXYR7leX2ma8wqwRDHzF/FUSoAguWOfiEM7DyixGZp4ifK25JEOacMc8Q1TMaOTmrQGvG28Uv8v6azzcGTstS9AyOLnQEgFpXTM8cM2/dvSh5YKep6ZBedEDMV0FHCvCZC9jQ01mC7Vw+wQ+1nXCWFSOHjEfVeKjAcYkjhrIZ4VLJ23LkhwLiJY/zkGWkl/+HS5w3otTFf2F3V+U2y/VmidjHX4zHmQ8HdLA4KFtb0ZsxozgnFdS4JeEK7WUBI55uUON61RGw6ObXrNYrP6Lei7LNNY3rcrOJGnNJunjc0zmEgV7vWNfskxd8nTtunx/Ozt4OhPKXEvCQ7UhCNjd/1XnS0lb/ai9FNF0+nMJ T4XisMvF F/JSPe/fac/iOSoFB9h9JVGSWrW2E1Q//NdDG2w/QkOw/b7xRX3PVMkWKpC/op4eQG0xINDQi7hQq41SyTA6fjl35FyHij6rVDYpl2/ZYxww/zeI/20DXKWimvxH0CLg/yWecKE/EUKEjkuIhoq/iFvs42s2V5XRhwwBhbZq9a8eoK0M3dwyAFGWImCSCP9NRkaVioAIKuGJ6JyYNPgk154eZyjLH9wKl2WZthYJEgt1ruj3Gm74r6zeVGEeCRh4Izfa/3FCfeySPHHss+dFC/lwDqfgojefTANO5ikZMi58Neds= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 8/8/23 12:05 AM, David Hildenbrand wrote: > On 07.08.23 14:41, David Hildenbrand wrote: >> On 07.08.23 14:27, Michal Hocko wrote: >>> On Sat 05-08-23 19:54:23, Aneesh Kumar K V wrote: >>> [...] >>>> Do you see a need for firmware-managed memory to be hotplugged in with >>>> different memory block sizes? >>> >>> In short. Yes. Slightly longer, a fixed size memory block semantic is >>> just standing in the way and I would even argue it is actively harmful. >>> Just have a look at ridicously small memory blocks on ppc. I do >>> understand that it makes some sense to be aligned to the memory model >>> (so sparsmem section aligned). In an ideal world, memory hotplug v2 >>> interface (if we ever go that path) should be physical memory range based. >> >> Yes, we discussed that a couple of times already (and so far nobody >> cared to implement any of that). >> >> Small memory block sizes are very beneficial for use cases like PPC >> dlar, virtio-mem, hyperv-balloon, ... essentially in most virtual >> environments where you might want to add/remove memory in very small >> granularity. I don't see that changing any time soon. Rather the opposite. >> >> Small memory block sizes are suboptimal for large machines where you >> might never end up removing such memory (boot memory), or when dealing >> with devices that can only be removed in one piece (DIMM/kmem). We >> already have memory groups in place to model that. >> >> For the latter it might be beneficial to have memory blocks of larger >> size that correspond to the physical memory ranges. That might also make >> a memmap (re-)configuration easier. >> >> Not sure if that is standing in any way or is harmful, though. >> > > Just because I thought of something right now, I'll share it, maybe it makes sense. > > Assume when we get add_memory*(MHP_MEMMAP_ON_MEMORY) and it is enabled by the admin: > > 1) We create a single altmap at the beginning of the memory > > 2) We create the existing fixed-size memory block devices, but flag them >    to be part of a single "altmap" unit. > > 3) Whenever we trigger offlining of a single such memory block, we >    offline *all* memory blocks belonging to that altmap, essentially >    using a single offline_pages() call and updating all memory block >    states accordingly. > > 4) Whenever we trigger onlining of a single such memory block, we >    online *all* memory blocks belonging to that altmap, using a single >    online_pages() call. > > 5) We fail remove_memory() if it doesn't cover the same (altmap) range. > > So we can avoid having a memory block v2 (and all that comes with that ...) for now and still get that altmap stuff sorted out. As that altmap behavior can be controlled by the admin, we should be fine for now. > > I think all memory notifiers should already be able to handle bigger granularity, but it would be easy to check. Some internal things might require a bit of tweaking. > We can look at the possibility of using the altmap space reserved for a namespace (via option -M dev) for allocating struct page memory even with dax/kmem. -aneesh