From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E85D2C77B7C for ; Wed, 25 Jun 2025 17:18:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 623528D000A; Wed, 25 Jun 2025 13:18:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5FB228D0001; Wed, 25 Jun 2025 13:18:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E9B28D000A; Wed, 25 Jun 2025 13:18:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3E2988D0001 for ; Wed, 25 Jun 2025 13:18:10 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B5821BA8ED for ; Wed, 25 Jun 2025 17:18:09 +0000 (UTC) X-FDA: 83594581098.22.F91DD1D Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf08.hostedemail.com (Postfix) with ESMTP id 4AD99160009 for ; Wed, 25 Jun 2025 17:18:07 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Hu8R2TwE; spf=pass (imf08.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750871887; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hJmiqIGVi9jYz2XV2RDTUFNp29tnqRL83h4+ynhToF0=; b=vnIn/oXiI7CQEfZVRd4nAQ7QTtRflfd0coLSofANcIKyAJnh0srvGoipYsEkuk3TOiQSgs ObiS4KZiGiPxTT+/a8UWrRLclVopt5wr16XnRFlpCUbE9N5mHucgibzzILVFyD9tbr6RjC 4PicM3J90HatIRgOXiFXF7GSZ1hkj40= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Hu8R2TwE; spf=pass (imf08.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750871887; a=rsa-sha256; cv=none; b=RSjADdjWSOiTESKpHWwtBFUG6w9pg3SgS6IAVUuIw9aTSqlYZlE6bzkMF57DYtQzGmc/F5 iBX2bx75rhAbPUVZHD1ncALHqm6cJXTbKR0WyLcT/9KC7NBCzJIYlipLptM18/lS46Gz0A 7/lwtxcjBQGbAubNatcR1fyFJppwSuA= Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PFUIrI003394; Wed, 25 Jun 2025 17:18:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=hJmiqI GVi9jYz2XV2RDTUFNp29tnqRL83h4+ynhToF0=; b=Hu8R2TwE+A8WyFG1N5BlSK wMQo3nhEMe5EbAwT4IiewOzMG+gtt+l/D8epBEZVektbDqDWNH6N0KoZ00258HhP yNiVyX7+WZkeWt6RBefwfhFibmbNG4ET6neyu6N9IdPwL8dZyIGwfUAub3gFtJa+ 5ZTr/+T39bsaFTpWrymI2IiSk8s6CE96c3/w7m5s5lrXSSeSB8m1nybYlpme08eY qNE48LFW0n4PZO+dUP+FYf3o4tG2RzOu9fz2C02aU7ks6PFQIA2mULaS3hZp+NAG KFNj3KOv+TXx33hZcWbhCxjJUXeq/sMlcCLqlBgWhb0AlnDenV4zd8ypTSvnv88A == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47dj5u0w7n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 17:18:01 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 55PHI00j030154; Wed, 25 Jun 2025 17:18:01 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47dj5u0w7f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 17:18:00 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 55PF1Uub004643; Wed, 25 Jun 2025 17:17:59 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 47e99kt8nw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 17:17:59 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 55PHHvkH44827114 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 25 Jun 2025 17:17:57 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9A56820043; Wed, 25 Jun 2025 17:17:57 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AE6F220040; Wed, 25 Jun 2025 17:17:53 +0000 (GMT) Received: from li-06431bcc-2712-11b2-a85c-a6fe68df28f9.ibm.com (unknown [9.124.208.75]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 25 Jun 2025 17:17:53 +0000 (GMT) Date: Wed, 25 Jun 2025 22:47:50 +0530 From: Donet Tom To: Dev Jain Cc: Lorenzo Stoakes , Aboorva Devarajan , akpm@linux-foundation.org, Liam.Howlett@oracle.com, shuah@kernel.org, pfalcato@suse.de, david@redhat.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, npache@redhat.com, ryan.roberts@arm.com, baohua@kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, ritesh.list@gmail.com Subject: Re: [PATCH 1/6] mm/selftests: Fix virtual_address_range test issues. Message-ID: References: <79bdd993-0e9c-4d7d-b42c-4b5750eff140@lucifer.local> <8e23c5d3-6ce3-4fe8-b6fe-69658d5d0727@lucifer.local> <815793f1-6800-4b9a-852e-f13d6308f50f@arm.com> <2756fa2b-e8bf-4c66-bf9b-c85dc63dfc33@lucifer.local> <41d9a70d-9791-4212-af23-5b13d8e4a47d@arm.com> <16fff6e9-98f5-4004-9906-feac49f0bbb4@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <16fff6e9-98f5-4004-9906-feac49f0bbb4@arm.com> X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: GwdDxKWbBi4q9Rqmr219TkpxO9foB5-d X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDEyOCBTYWx0ZWRfX+cC+8RLbZJWH wEP3loW+cj4DbnfX/wab4EtH45pdwH1Szs+gPGN7IFRsblju3yLjvMky1sFDpzHRbD1ZvKXjChc EkGDIS4/ygaDFkPxq8yFu3X4cSWUcgs3zke4ronSLVNmR1YnD+CiDCobzAP1PNt/Sp+6wjyJ+44 YN2JwUE9qDkqar9MC3N3k1NQQjKqAl5oDt3GcvUARZCaz0SaOx5JYYSUN4AmrF2l8T2MqFDv46x pDwXvEjERfnGHcmvIZwx3BkU2gL0HDz8xATGsuY6C5nXZNVBHjVok71/5DqO2eifX79G5Nwkw+D c2NRN2bOQ2NytKK4jxpkN4oVGa2QIKURLzEA4iQfPWyTQm0o5Qpw5ghyVc5C2pCsk2+5k2Josqp slm8qUidDYRz/clr7OHjf5pJqOTSPEnaOiGnKPJ6vgF7CTK3UkCSs1Ou9mX2tzVAPBf+0D0H X-Authority-Analysis: v=2.4 cv=MshS63ae c=1 sm=1 tr=0 ts=685c2f49 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=BaPsrneplRJP-Z4xSb0A:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: ai7n1ObA81d7M5cF_SeK_zdL-urTLJWf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_05,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 bulkscore=0 lowpriorityscore=0 spamscore=0 mlxlogscore=999 impostorscore=0 clxscore=1015 phishscore=0 malwarescore=0 suspectscore=0 adultscore=0 priorityscore=1501 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2506250128 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 4AD99160009 X-Stat-Signature: 3uqermtu3q68ujbstme75wuew1mb8wnf X-Rspam-User: X-HE-Tag: 1750871887-190974 X-HE-Meta: U2FsdGVkX18zeNtv7FEXzj1NPUWMj0gX245ZVwiFWBCLEOWjfrR+fk4q8zR9mCV9CbEoo3oiUwNEFqMhurF+gOaVmPVwWQCf/cKhAnVkByKQOiitBhSsTdoH+N17xiEzWT04gsqra93Plww39wkBoXr0s8qV1kFq0Is+WUGW3jVL7Zxbew1x7DojswhwoLR30hpBeNgXUsZgRNhY35r+hIiP8vKephTlMI+IbnEBDBx+IIVWv4BvcgXrbFJss/JGR52Hz+50KbL5GiQnHRgsype3eliXG+2kNP9Udlazgz0J0vi/wWFr65Pp/tTq4qmpbwXcdAxwGdd0UHtkLRykYqIztOH8pZy0vpVWxRQa0lBvNmp5IBXPnMW+IuQcJxP46ZnmLEWPHlNnUGIZByI4ThaGJY75Mq0yFYbzNJX2hysEAtNnK1GQTuievS1w1Ll0FeSMskssi+WrGoPevXgLnVE05X1TmDMhxCZsbnSpk2tzhhBevPsZMWmuVrcrtANTQmedIWf47/T8YEug+sWrc199cWgzAPyCY0TbEUuCHQEVLjXFqi4skoFMak0cJKyXpJWJNb9mxyr0K0E1u9LWU2aQ+52h1kUXNQXgExv/jS1OxG4+QgfegvoupP/brZZ7b5X5l5dXuzNOqg8h0o8aKRblKq6sa4X2xiQKQV38r6erfArgXpNyJTSoH0FXOEUeVBeCkt3CLwq5U+4aU/E7g0hPT58jOeD2gDpoie/u+MPxfknGsSvyvOdN1QgNRxCz4TpnJcvPJw2vXRQg/e3PS1vlvMUVOuFKf6DsEm15C0Zznq6WubIGiOIU2zjxQUNs0Q29+bkKbAaNRe2uqcBbO0SQmmYcGnLQeCmTHGUrlUiZg8/BNDo783UtGtzufFe2CK2k+5WwdmiZJY7NQXCgazkdFk5tVdn149cbNEnPVyKFyipgWtME/f/RlJRfe/C0zFgxm04YA9BZIIV/Z/W h9qEdncW 13ElSHzR5ZdXlKALWYjcUVm5fKV/jbJWOGuCsFxMn9dnS7OAZoawlz3+AeE2FD4RbjRSYQP0N2VrGu3Dz92QfdwX8t2cvcmqdFjyj5+Ya44Nyiq3722hRuNZ7ZBLYRpsrSIS9uSKw4VKNCUXFIOs5Y4xVykiN0Xi11iC63iorjyusndYRXs4L/oG7uG7EM9Syj/wABhPMd7bJu/qjg/q9Ol4Nq/fzii4YjpnU+ieYoTyn/lx0TI1EDMf3wcEMKkiwZQ9UXCDAwJSMe1iwG3+e6a8ctEp3REuVzsbF6x4JSNwnnNauCorrBXb+0/rDJalb1oUBJRcKL9ZDSN4lgq0MUPr7D7Orm3pNnkbd16U0WcBXjRlZq02J+rZZ4Qq3AaEf+dV2YT4zOYBMp2kiVnE9vXtumltwdthWsz8NoCZLtXPHGySrcvywhTCBsN/9la9bk5eJcaeba3bc7uo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 25, 2025 at 06:22:53PM +0530, Dev Jain wrote: > > On 19/06/25 1:53 pm, Donet Tom wrote: > > On Wed, Jun 18, 2025 at 08:13:54PM +0530, Dev Jain wrote: > > > On 18/06/25 8:05 pm, Lorenzo Stoakes wrote: > > > > On Wed, Jun 18, 2025 at 07:47:18PM +0530, Dev Jain wrote: > > > > > On 18/06/25 7:37 pm, Lorenzo Stoakes wrote: > > > > > > On Wed, Jun 18, 2025 at 07:28:16PM +0530, Dev Jain wrote: > > > > > > > On 18/06/25 5:27 pm, Lorenzo Stoakes wrote: > > > > > > > > On Wed, Jun 18, 2025 at 05:15:50PM +0530, Dev Jain wrote: > > > > > > > > Are you accounting for sys.max_map_count? If not, then you'll be hitting that > > > > > > > > first. > > > > > > > run_vmtests.sh will run the test in overcommit mode so that won't be an issue. > > > > > > Umm, what? You mean overcommit all mode, and that has no bearing on the max > > > > > > mapping count check. > > > > > > > > > > > > In do_mmap(): > > > > > > > > > > > > /* Too many mappings? */ > > > > > > if (mm->map_count > sysctl_max_map_count) > > > > > > return -ENOMEM; > > > > > > > > > > > > > > > > > > As well as numerous other checks in mm/vma.c. > > > > > Ah sorry, didn't look at the code properly just assumed that overcommit_always meant overriding > > > > > this. > > > > No problem! It's hard to be aware of everything in mm :) > > > > > > > > > > I'm not sure why an overcommit toggle is even necessary when you could use > > > > > > MAP_NORESERVE or simply map PROT_NONE to avoid the OVERCOMMIT_GUESS limits? > > > > > > > > > > > > I'm pretty confused as to what this test is really achieving honestly. This > > > > > > isn't a useful way of asserting mmap() behaviour as far as I can tell. > > > > > Well, seems like a useful way to me at least : ) Not sure if you are in the mood > > > > > to discuss that but if you'd like me to explain from start to end what the test > > > > > is doing, I can do that : ) > > > > > > > > > I just don't have time right now, I guess I'll have to come back to it > > > > later... it's not the end of the world for it to be iffy in my view as long as > > > > it passes, but it might just not be of great value. > > > > > > > > Philosophically I'd rather we didn't assert internal implementation details like > > > > where we place mappings in userland memory. At no point do we promise to not > > > > leave larger gaps if we feel like it :) > > > You have a fair point. Anyhow a debate for another day. > > > > > > > I'm guessing, reading more, the _real_ test here is some mathematical assertion > > > > about layout from HIGH_ADDR_SHIFT -> end of address space when using hints. > > > > > > > > But again I'm not sure that achieves much and again also is asserting internal > > > > implementation details. > > > > > > > > Correct behaviour of this kind of thing probably better belongs to tests in the > > > > userland VMA testing I'd say. > > > > > > > > Sorry I don't mean to do down work you've done before, just giving an honest > > > > technical appraisal! > > > Nah, it will be rather hilarious to see it all go down the drain xD > > > > > > > Anyway don't let this block work to fix the test if it's failing. We can revisit > > > > this later. > > > Sure. @Aboorva and Donet, I still believe that the correct approach is to elide > > > the gap check at the crossing boundary. What do you think? > > > > > One problem I am seeing with this approach is that, since the hint address > > is generated randomly, the VMAs are also being created at randomly based on > > the hint address.So, for the VMAs created at high addresses, we cannot guarantee > > that the gaps between them will be aligned to MAP_CHUNK_SIZE. > > > > High address VMAs > > ----------------- > > 1000000000000-1000040000000 r--p 00000000 00:00 0 > > 2000000000000-2000040000000 r--p 00000000 00:00 0 > > 4000000000000-4000040000000 r--p 00000000 00:00 0 > > 8000000000000-8000040000000 r--p 00000000 00:00 0 > > e80009d260000-fffff9d260000 r--p 00000000 00:00 0 > > > > I have a different approach to solve this issue. > > > > From 0 to 128TB, we map memory directly without using any hint. For the range above > > 256TB up to 512TB, we perform the mapping using hint addresses. In the current test, > > we use random hint addresses, but I have modified it to generate hint addresses linearly > > starting from 128TB. > > > > With this change: > > > > The 0–128TB range is mapped without hints and verified accordingly. > > > > The 128TB–512TB range is mapped using linear hint addresses and then verified. > > > > Below are the VMAs obtained with this approach: > > > > 10000000-10010000 r-xp 00000000 fd:05 135019531 > > 10010000-10020000 r--p 00000000 fd:05 135019531 > > 10020000-10030000 rw-p 00010000 fd:05 135019531 > > 20000000-10020000000 r--p 00000000 00:00 0 > > 10020800000-10020830000 rw-p 00000000 00:00 0 > > 1004bcf0000-1004c000000 rw-p 00000000 00:00 0 > > 1004c000000-7fff8c000000 r--p 00000000 00:00 0 > > 7fff8c130000-7fff8c360000 r-xp 00000000 fd:00 792355 > > 7fff8c360000-7fff8c370000 r--p 00230000 fd:00 792355 > > 7fff8c370000-7fff8c380000 rw-p 00240000 fd:00 792355 > > 7fff8c380000-7fff8c460000 r-xp 00000000 fd:00 792358 > > 7fff8c460000-7fff8c470000 r--p 000d0000 fd:00 792358 > > 7fff8c470000-7fff8c480000 rw-p 000e0000 fd:00 792358 > > 7fff8c490000-7fff8c4d0000 r--p 00000000 00:00 0 > > 7fff8c4d0000-7fff8c4e0000 r-xp 00000000 00:00 0 > > 7fff8c4e0000-7fff8c530000 r-xp 00000000 fd:00 792351 > > 7fff8c530000-7fff8c540000 r--p 00040000 fd:00 792351 > > 7fff8c540000-7fff8c550000 rw-p 00050000 fd:00 792351 > > 7fff8d000000-7fffcd000000 r--p 00000000 00:00 0 > > 7fffe9c80000-7fffe9d90000 rw-p 00000000 00:00 0 > > 800000000000-2000000000000 r--p 00000000 00:00 0 -> High Address (128TB to 512TB) > > > > diff --git a/tools/testing/selftests/mm/virtual_address_range.c b/tools/testing/selftests/mm/virtual_address_range.c > > index 4c4c35eac15e..0be008cba4b0 100644 > > --- a/tools/testing/selftests/mm/virtual_address_range.c > > +++ b/tools/testing/selftests/mm/virtual_address_range.c > > @@ -56,21 +56,21 @@ > > #ifdef __aarch64__ > > #define HIGH_ADDR_MARK ADDR_MARK_256TB > > -#define HIGH_ADDR_SHIFT 49 > > +#define HIGH_ADDR_SHIFT 48 > > #define NR_CHUNKS_LOW NR_CHUNKS_256TB > > #define NR_CHUNKS_HIGH NR_CHUNKS_3840TB > > #else > > #define HIGH_ADDR_MARK ADDR_MARK_128TB > > -#define HIGH_ADDR_SHIFT 48 > > +#define HIGH_ADDR_SHIFT 47 > > #define NR_CHUNKS_LOW NR_CHUNKS_128TB > > #define NR_CHUNKS_HIGH NR_CHUNKS_384TB > > #endif > > -static char *hint_addr(void) > > +static char *hint_addr(int hint) > > { > > - int bits = HIGH_ADDR_SHIFT + rand() % (63 - HIGH_ADDR_SHIFT); > > + unsigned long addr = ((1UL << HIGH_ADDR_SHIFT) + (hint * MAP_CHUNK_SIZE)); > > - return (char *) (1UL << bits); > > + return (char *) (addr); > > } > > static void validate_addr(char *ptr, int high_addr) > > @@ -217,7 +217,7 @@ int main(int argc, char *argv[]) > > } > > for (i = 0; i < NR_CHUNKS_HIGH; i++) { > > - hint = hint_addr(); > > + hint = hint_addr(i); > > hptr[i] = mmap(hint, MAP_CHUNK_SIZE, PROT_READ, > > MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); > > Ah you sent it here, thanks. This is fine really, but the mystery is > something else. > Thanks Dev I can send out v2 with this patch included, right? > > > > > > > Can we fix it this way? >