From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 733ACC71145 for ; Thu, 24 Aug 2023 10:52:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4963280071; Thu, 24 Aug 2023 06:52:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF7B7280070; Thu, 24 Aug 2023 06:52:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9870280071; Thu, 24 Aug 2023 06:52:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A664B280070 for ; Thu, 24 Aug 2023 06:52:49 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7DC421A02B5 for ; Thu, 24 Aug 2023 10:52:49 +0000 (UTC) X-FDA: 81158685258.29.87ED13E Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf01.hostedemail.com (Postfix) with ESMTP id C5C4040015 for ; Thu, 24 Aug 2023 10:52:46 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=CrcLERCT; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf01.hostedemail.com: domain of jaypatel@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=jaypatel@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692874367; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/iBssC1JiV9s6mqIwqwxT25qjhIa+NdaaLllj05cma8=; b=1jrB1/4/5FRIcHgKCeunodbYtDuPy+mdqPdl8ZDHR59V2S7rm8uIUjAwoNyeN0kRjpjCjk vNpP4+OL5JlNi5M5Fdo97hsMB3F73eL5PQ5Mobx0oMjVCa5jY0gXaVxRVzN36xxyK4aNPW UPgtwWT5R2B0g91PPdQWiJxIx887Sdk= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=CrcLERCT; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf01.hostedemail.com: domain of jaypatel@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=jaypatel@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692874367; a=rsa-sha256; cv=none; b=5vURk1ravvrLQ2H8Q6fYfFgPu73FRa/E2mfGeWwCrf/fbaOiU0Z5HtoHtUosO/lhrmu+rq WddQ3OMtnyZWjNd58iPCqLCkU3jiU268NRrwpQK0X1jMGX30goIrbeNqVr9muZ5g5clQeI J3kestm1HdtYP7X3/iI7OVZzmBxqg7Y= Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37OAgEgN002121; Thu, 24 Aug 2023 10:52:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : reply-to : to : cc : date : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=pp1; bh=/iBssC1JiV9s6mqIwqwxT25qjhIa+NdaaLllj05cma8=; b=CrcLERCT5+z4rC/0BSdfUWDGFRFK/+D/x8iDk7OJHu53coBMTmYdPODOJBy6wFpLJvA7 6Riw8+IthE9OeA1+WpfeVa7sPGaQz/e1wJYf+7n6A/CACfOEAp0h97Ru5W96LWiO+ZTT BmUN8+/sAYEsHD40y0JY1t3eCJzZTcpyQSInxGJg2XnDsKrFaS659epx5G0f0KIsa9XN yGr51LupojVx/YUYrIhswq7RAZgS4DeNQs82FFgPjtQkUa6XO+l5NHCTPB1UYZq7VoY4 EbWTRRI6lcSbh4LLK2xhkSG60TDBx6cOMQdqSJS6KqStxTRkcU5JU5eCKoQF3IUJElWa ZA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3sp5p9r7hq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Aug 2023 10:52:42 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 37OAgKSn002282; Thu, 24 Aug 2023 10:52:41 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3sp5p9r7h4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Aug 2023 10:52:41 +0000 Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 37O9sVGB016435; Thu, 24 Aug 2023 10:52:40 GMT Received: from smtprelay04.dal12v.mail.ibm.com ([172.16.1.6]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3sn227xb8p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Aug 2023 10:52:40 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay04.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 37OAqdW115074034 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Aug 2023 10:52:39 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B67F5805D; Thu, 24 Aug 2023 10:52:39 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3605358059; Thu, 24 Aug 2023 10:52:35 +0000 (GMT) Received: from patel.in.ibm.com (unknown [9.109.195.155]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 24 Aug 2023 10:52:34 +0000 (GMT) Message-ID: <7fdf3f5dfd9fa1b5e210cc4176cac58a9992ecc0.camel@linux.ibm.com> Subject: Re: [RFC PATCH v4] mm/slub: Optimize slub memory usage From: Jay Patel Reply-To: jaypatel@linux.ibm.com To: Vlastimil Babka , Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, tsahu@linux.ibm.com, piyushs@linux.ibm.com Date: Thu, 24 Aug 2023 16:22:33 +0530 In-Reply-To: References: <20230720102337.2069722-1-jaypatel@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 (3.28.5-22.el8) X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: NyX_iP0oIBM9g28kYI_m3X4WxTQ8jCj5 X-Proofpoint-GUID: feJ0GcYknW5Bob7JYROqu7wpd3R0ZUXq Content-Transfer-Encoding: 7bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-24_06,2023-08-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 clxscore=1011 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 spamscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2308240085 X-Rspam-User: X-Stat-Signature: pw7in1hs5jjdzgcppgeq79ckiud91ec8 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C5C4040015 X-HE-Tag: 1692874366-496005 X-HE-Meta: U2FsdGVkX197g7aGMHaeXSz2jWtO63lZKxbvTqstLoGTX3D2mlPP8TL2wXc40tHoWvuYmCBX6cGcsrE6/9v1yQ/ZHoIRPBgDKtGfYgapfV7ZQ640N0a0WQqZEU7jZnwkJncZLL8lr1Rs611hO126lvSm4J2IkiSCKA/ACI2j5zeYqplXg0sFLyF2lzMZu8h/plG2j+gLULy0aPbAIgorUs9jeaRmvfne6ozU7+Acq5nSJIpSScc0y7roo2kJReeey7bnyVrpN4PNKPZGvmXCS/Mfw+If8PhRO+2WPy42K7v+nnOJXmJc8RS/vbWXb4XOu01wR8I2Had1uxPfCtRQluhJ01VC3p9WnIkTa1CsZIP0eMMpwjfO+O9WbwfTmgVb94PKDKgz1rvggRisqvTFficLrUmBxoDnqHfeqdnQF8n83CxDeIgaRffeXqBNxQ81tyyboY615pEXCXJOpdX3W0timS4J8US1Iq5CUADGC6zvfBtTqFYh4u0p1/oGshZnr8Ot7VqqjyJnr7rjgYIZYCchlBSMunJFzNeJjMBNLdmXcVZLm3RJ/9ydFldnGjVfV4K1kriIxUQUymtMLLYALc0Rlblm/YDGNhfqDj6SxAojN/cibYCwtCEBS61ihPOSfiIlkY10I24e70Sq3rLboHDIjEMjBdRw7kbqKtAfY33Bzt0fDaQmOQ8Zv0de4XyPVwARHrKqQE8XwnpNCp6wGPz1ebkhatvCsg1DZS9Rqe0n/aMgKz+lR88j3phDX67OCchw6BJR9cLKFluIzgehniEvI/NUJBZkGeqo2FiJqbvhStjA7aanLF0WO9lFXV2UY6DTO2Qula5qSBPqwl+T8W/bBUIMFBJV/M7ws5y8G23/F/KHUXMboQWcIDyHfHm56MQ36FDg6HMcxPt41jIHUsD175iBtyjyyHzgE8HL5Y3wWGPAp96MbWzeGzTAUseTc+h5f+8wZjk+Y4uxb+8 H4pPi0iN AZ+7WTXL+bXZ+f4gmUUMVpRI1oRxFTe5irQ7dzVdH3fTyJve9wnv1QevJQfQQu/AGRoprwKpfinNNrYW5e2fHbdXlTn/EHHwBk3brWkl6Qum1voD+y/ZwgpGUOOR8+hVzA0FUzXNMniLUGJZcUqauZKp7F3mNQCYKFO5QmXMuBoMUzuMnNqTcexDybWAxyUKKMiOwXvDVhwoUZkychVMW4Wz6nOudCbrGIxPGLSykQagqYIuK2LIycCOJt7Yyhh/7Rbh1/Spt1Q9uTxLkqMLNZrHqiBHDV9GTo7Xp5tTasuunu25uUKSN3/9LeA4D7NHikOGH4WzdVo+AZRV8Y6pLbobjsfAR/nWweEPahp7TAtAjq0o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 2023-08-11 at 17:43 +0200, Vlastimil Babka wrote: > On 8/10/23 19:54, Hyeonggon Yoo wrote: > > > order = calc_slab_order(size, > > > min_objects, > > > slub_max_order, > > > fraction); > > > @@ -4159,14 +4164,6 @@ static inline int calculate_order(unsigned > > > int size) > > > min_objects--; > > > } > > > - /* > > > - * We were unable to place multiple objects in a slab. > > > Now > > > - * lets see if we can place a single object there. > > > - */ > > > - order = calc_slab_order(size, 1, slub_max_order, 1); > > > - if (order <= slub_max_order) > > > - return order; > > > > I'm not sure if it's okay to remove this? > > It was fine in v2 because the least wasteful order was chosen > > regardless of fraction but that's not true anymore. > > > > Otherwise, everything looks fine to me. I'm too dumb to anticipate > > the outcome of increasing the slab order :P but this patch does not > > sound crazy to me. > > I wanted to have a better idea how the orders change so I hacked up a > patch > to print them for all sizes up to 1MB (unnecessarily large I guess) > and also > for various page sizes and nr_cpus (that's however rather invasive > and prone > to me missing some helper being used that still relies on real > PAGE_SHIFT), > then I applied v4 (needed some conflict fixups with my hack) on top: > > https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slab-orders > > As expected, things didn't change with 4k PAGE_SIZE. With 64k > PAGE_SIZE, I > thought the patch in v4 form would result in lower orders, but seems > not always? > > I.e. I can see before the patch: > > Calculated slab orders for page_shift 16 nr_cpus 1: > 8 0 > 4376 1 > > (so until 4368 bytes it keeps order at 0) > > And after: > 8 0 > 2264 1 > 2272 0 > 2344 1 > 2352 0 > 2432 1 > > Not sure this kind of "oscillation" is helpful with a small machine > (1CPU), > and 64kB pages so the unused part of page is quite small. > Hi Vlastimil, With patch. it will cause the fraction_size to rise to 32 when utilizing a 64k page size. As a result, the maximum wastage cap for each slab cache will be 2k (64k divided by 32). Any object size exceeding this cap will be moved to order 1 or beyond due to which this oscillation is seen. > With 16 cpus, AFAICS the orders are also larger for some sizes. > Hm but you reported reduction of total slab memory which suggests > lower > orders were selected somewhere, so maybe I did some mistake.A AFAIK total slab memory is reduce because of two reason (with this patch for larger page size) 1) order for some slab cache is reduce (by increasing fraction_size) 2) Have also seen reduction in overall slab cache numbers as because of increasing page order > > Anyway my point here is that this evaluation approach might be > useful, even > if it's a non-upstreamable hack, and some postprocessing of the > output is > needed for easier comparison of before/after, so feel free to try > that out. Thank you for this details test :) > > BTW I'll be away for 2 weeks from now, so further feedback will have > to come > from others in that time... > Do we have any additional feedback from others on the same matter? Thank Jay Patel > > Thanks! > > -- > > Hyeonggon