From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 655C4C0015E for ; Fri, 11 Aug 2023 06:52:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5EFA6B0071; Fri, 11 Aug 2023 02:52:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C0EB26B0072; Fri, 11 Aug 2023 02:52:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD76C6B0074; Fri, 11 Aug 2023 02:52:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A0B1C6B0071 for ; Fri, 11 Aug 2023 02:52:45 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 6FA371610CF for ; Fri, 11 Aug 2023 06:52:45 +0000 (UTC) X-FDA: 81110905890.05.5D9EA68 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf20.hostedemail.com (Postfix) with ESMTP id 9625D1C0017 for ; Fri, 11 Aug 2023 06:52:42 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=UGzn1NHG; spf=pass (imf20.hostedemail.com: domain of jaypatel@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=jaypatel@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691736763; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pglKfzeTzH7C8JmN/JLn16kZrlqlybWDwPpwY7+ZMJo=; b=2OdeIQYI9wVV5SBqOgXBc4HEXjTv7FDmFi0LYgXkFH8VIosLKx9fdxvK0WmC43FR7giRM4 X3yb79YVhCCzMXZTMgagS9y7jOGSVkw/Cs7eF0B1p2ij7b8h7SbarGwWmu9AqC6U9k9BNM nIjx7GxK2zy/4GNNYmSZPhyx9MP5GPw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691736763; a=rsa-sha256; cv=none; b=R3T1VREh8Og46Wwxr9DUkhcJzEfuM0GHq/9gzBDRswFtqDatF6UmAyoynLOoQHwE9P6qeN +tB5EHHpW5piMZ3/BlqHADi9n6AnpfV94CYeGIApyVnXrqAOm/qzR2penOU9gK1HU0mJjD 6nz6YDVe5k5Ron0gYYPbzW3nFn3WMiQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=UGzn1NHG; spf=pass (imf20.hostedemail.com: domain of jaypatel@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=jaypatel@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37B6aJph014984; Fri, 11 Aug 2023 06:52:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : reply-to : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=pglKfzeTzH7C8JmN/JLn16kZrlqlybWDwPpwY7+ZMJo=; b=UGzn1NHG6sVxDd0efoO4divXY9JWWIb2Dy1NkSKM4GvCW8m8Xns4bFNVV/HoNRGNUNjv ZU5toYP0CHd6qbqfWEBdfHMt1Uu1p2mGimLJzGADEsfVayB98qZR07KAUNivi2aeQGPr vmtN4oziNOzc3P/Uv6g8PSG2sS6sK/QHDpBWD/XSApQWjbh4DbBONsXB74qVL7yx649L dQxUOMhNwPyY5cLiPqvs/CRTXUeYyF3YC8G3U7Nxo+8gQ18vTvNVDMjPt4fMpoIykne0 sjU8O6gCYeYKSVb/33M9XLr+W39mhVAdYo+pT+IhhSoH7pC6PCNIXo4/2Kvvb9s2r3oe qw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3sdfp0gkhe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Aug 2023 06:52:38 +0000 Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 37B6c3nH023176; Fri, 11 Aug 2023 06:52:38 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3sdfp0gkh1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Aug 2023 06:52:37 +0000 Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 37B5VZad001772; Fri, 11 Aug 2023 06:52:37 GMT Received: from smtprelay02.wdc07v.mail.ibm.com ([172.16.1.69]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 3sa3f2g0yv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Aug 2023 06:52:36 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay02.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 37B6qZ9R66060580 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 11 Aug 2023 06:52:35 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9C1B958058; Fri, 11 Aug 2023 06:52:35 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D17E658059; Fri, 11 Aug 2023 06:52:31 +0000 (GMT) Received: from patel (unknown [9.61.51.89]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 11 Aug 2023 06:52:31 +0000 (GMT) Message-ID: Subject: Re: [RFC PATCH v4] mm/slub: Optimize slub memory usage From: Jay Patel Reply-To: jaypatel@linux.ibm.com To: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, aneesh.kumar@linux.ibm.com, tsahu@linux.ibm.com, piyushs@linux.ibm.com Date: Fri, 11 Aug 2023 12:22:30 +0530 In-Reply-To: References: <20230720102337.2069722-1-jaypatel@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 (3.28.5-22.el8) Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 6ULmPu6WAb2k6axro-8HZ2-xA9n2MRmu X-Proofpoint-ORIG-GUID: ZB9rK4fVq6u1PnpQg4a3Lp6rGfHs7I3u X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-08-10_20,2023-08-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 bulkscore=0 adultscore=0 spamscore=0 impostorscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2308110059 X-Stat-Signature: ejg87zhrnxd9soxhmpr9z3bjo11ak8sc X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 9625D1C0017 X-Rspam-User: X-HE-Tag: 1691736762-521231 X-HE-Meta: U2FsdGVkX18IMmd5vlwC9ab33tGTB+VQV4g7yhTlQYy0okY8w29kkeenBid3HgJ+1refsuQPnoYy2GeOjV0z0B2p6CAdfJBtHxFz649e+adDyjukP+VS8lwM0rY62HQUW9dcpzfF15gn8VOb7QNQQ1pA9dnpkTXdZw1vp+QB0xd5Sj/jejq2sdbGkzcaH9liFj7wyikHq9r4n0PELyPFSOxkzZdlruGDBTafSUeyD5WaX0Tg4Kx62vaaYYQ4dfenNg9p6HyICdLy5GfzuYknGYa6iAlXfN561X/cgGqiCIusuu3rSzx1SKvp2Wl94ZIb2hB4+ETl6/uh31qxL6ynkXgGG06L2xHJOVKr94HRRX9Z8bF6gbcvSUJ2g0U41ORDW73lNSC+Kn07y3EQQzqU4bKsqBGioYNygFRCc4s2k3Gt85suAwZM2C6GLZR10ugYyYm3tlWSF+sObD60W/vW5ian3ukrBUXbMjskxDHhMx2oU+TzInA4fYhIqB7em4ZjY1gFMI4G8hUzIf6N3spkCHgDniUheKFy+mXZyfn6GROXatNf6xwQDZWtuc/ABB4qnY2siCpLRf/nd5rMepfRojg1VdTI3atG1xLLyhqvO/9fuAHSCmZOOkXcQsbtglRBM6TFpSkZdR+MfsDPsLwqNB4jpXCm9FM9g8d57tdowIjpJzp/MNd8eXSGLWG2XluDOlNxcVzidcnQ+3suCzPii6Jt47849pN2pjhomatS4ffBVKyQevMfFeP7K/7FFn4RTxeCGQnz/pE8kiJScsvpVqt8QB6Vmx+BoPCFxGP4pg1KIL7g7VgSOI4PBZi4Gd0hVIcVrAeaJL5/qe4UMtg9I35Rcmfu/1se9sBGCf8FZqt8u5S24Vt5DWB3zEJslfgdUrdh0dDpRdqI6MZ2c6w4QAYvNIExyRB2ZJbQ40uVOUCaVDlhCMVzivrAy6OVmwm49zicNmjtP2D8vSqLpu3 DqSqn9Nh 7wgopeQOC3F4kccaOPiN6PBc9OuQ3/nd4rhzIhKR+sLCAG9QiPY76qA+t6Oj6cwu42ElnSOr1HFe3AE7a4gMqr/1bbH84NsiL6sJuBx+drTDB/irAooYn0cUd3CU79Ijrj8i+ZTORkut+Z9KxlRR3EO24Uiwx1KROKPndLtJWISBvgSeqvSIG6e4UwrcZAmHyQVx7VwHkxj1nZKak+11K+wjXMPB2C72xqkWCcePksyivtoOEQch2Cu17a4jvJn/Qt5CXIhiyOngq/6cbOHMQln7//nB1avNQQy4OqHPwP9ic+kGCBCbtBsDeW/EqAK3aqSIP3z1Q0jB65nNUUi0jE9BOou4Gp7jnIeftLkvr9LE9pMc3DnoEYpWnpNyNRjSQbG8VXNNtP/mba1jOlmIyrfnDyi5elGpMPnYCQcBXc+h268g4cW2jkbd60Rc2zC/1SvqBAF2qPw3AAxKFBgblZB+f2hqM7skME1ZYs0dQ+856YQU+3XecpW1apA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 2023-08-11 at 02:54 +0900, Hyeonggon Yoo wrote: > On Thu, Jul 20, 2023 at 7:24 PM Jay Patel > wrote: > > In the current implementation of the slub memory allocator, the > > slab > > order selection process follows these criteria: > > > > 1) Determine the minimum order required to serve the minimum number > > of > > objects (min_objects). This calculation is based on the formula > > (order > > = min_objects * object_size / PAGE_SIZE). > > 2) If the minimum order is greater than the maximum allowed order > > (slub_max_order), set slub_max_order as the order for this slab. > > 3) If the minimum order is less than the slub_max_order, iterate > > through a loop from minimum order to slub_max_order and check if > > the > > condition (rem <= slab_size / fract_leftover) holds true. Here, > > slab_size is calculated as (PAGE_SIZE << order), rem is (slab_size > > % > > object_size), and fract_leftover can have values of 16, 8, or 4. If > > the condition is true, select that order for the slab. > > > > > > However, in point 3, when calculating the fraction left over, it > > can > > result in a large range of values (like 1 Kb to 256 bytes on 4K > > page > > size & 4 Kb to 16 Kb on 64K page size with order 0 and goes on > > increasing with higher order) when compared to the remainder (rem). > > This > > can lead to the selection of an order that results in more memory > > wastage. To mitigate such wastage, we have modified point 3 as > > follows: > > To adjust the value of fract_leftover based on the page size, while > > retaining the current value as the default for a 4K page size. > > > > Test results are as follows: > > > > 1) On 160 CPUs with 64K Page size > > > > +-----------------+----------------+----------------+ > > > Total wastage in slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot |After Hackbench | > > > Normal | 932 Kb | 1812 Kb | > > > With Patch | 729 Kb | 1636 Kb | > > > Wastage reduce | ~22% | ~10% | > > +-----------------+----------------+----------------+ > > > > +-----------------+----------------+----------------+ > > > Total slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 1855296 | 2944576 | > > > With Patch | 1544576 | 2692032 | > > > Memory reduce | ~17% | ~9% | > > +-----------------+----------------+----------------+ > > > > hackbench-process-sockets > > +-------+-----+----------+----------+-----------+ > > > Amean | 1 | 1.2727 | 1.2450 | ( 2.22%) | > > > Amean | 4 | 1.6063 | 1.5810 | ( 1.60%) | > > > Amean | 7 | 2.4190 | 2.3983 | ( 0.86%) | > > > Amean | 12 | 3.9730 | 3.9347 | ( 0.97%) | > > > Amean | 21 | 6.9823 | 6.8957 | ( 1.26%) | > > > Amean | 30 | 10.1867 | 10.0600 | ( 1.26%) | > > > Amean | 48 | 16.7490 | 16.4853 | ( 1.60%) | > > > Amean | 79 | 28.1870 | 27.8673 | ( 1.15%) | > > > Amean | 110 | 39.8363 | 39.3793 | ( 1.16%) | > > > Amean | 141 | 51.5277 | 51.4907 | ( 0.07%) | > > > Amean | 172 | 62.9700 | 62.7300 | ( 0.38%) | > > > Amean | 203 | 74.5037 | 74.0630 | ( 0.59%) | > > > Amean | 234 | 85.6560 | 85.3587 | ( 0.35%) | > > > Amean | 265 | 96.9883 | 96.3770 | ( 0.63%) | > > > Amean | 296 | 108.6893 | 108.0870 | ( 0.56%) | > > +-------+-----+----------+----------+-----------+ > > > > 2) On 16 CPUs with 64K Page size > > > > +----------------+----------------+----------------+ > > > Total wastage in slub memory | > > +----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 273 Kb | 544 Kb | > > > With Patch | 260 Kb | 500 Kb | > > > Wastage reduce | ~5% | ~9% | > > +----------------+----------------+----------------+ > > > > +-----------------+----------------+----------------+ > > > Total slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 275840 | 412480 | > > > With Patch | 272768 | 406208 | > > > Memory reduce | ~1% | ~2% | > > +-----------------+----------------+----------------+ > > > > hackbench-process-sockets > > +-------+----+---------+---------+-----------+ > > > Amean | 1 | 0.9513 | 0.9250 | ( 2.77%) | > > > Amean | 4 | 2.9630 | 2.9570 | ( 0.20%) | > > > Amean | 7 | 5.1780 | 5.1763 | ( 0.03%) | > > > Amean | 12 | 8.8833 | 8.8817 | ( 0.02%) | > > > Amean | 21 | 15.7577 | 15.6883 | ( 0.44%) | > > > Amean | 30 | 22.2063 | 22.2843 | ( -0.35%) | > > > Amean | 48 | 36.0587 | 36.1390 | ( -0.22%) | > > > Amean | 64 | 49.7803 | 49.3457 | ( 0.87%) | > > +-------+----+---------+---------+-----------+ > > > > Signed-off-by: Jay Patel > > --- > > Changes from V3 > > 1) Resolved error and optimise logic for all arch > > > > Changes from V2 > > 1) removed all page order selection logic for slab cache base on > > wastage. > > 2) Increasing fraction size base on page size (keeping current > > value > > as default to 4K page) > > > > Changes from V1 > > 1) If min_objects * object_size > PAGE_ALLOC_COSTLY_ORDER, then it > > will return with PAGE_ALLOC_COSTLY_ORDER. > > 2) Similarly, if min_objects * object_size < PAGE_SIZE, then it > > will > > return with slub_min_order. > > 3) Additionally, I changed slub_max_order to 2. There is no > > specific > > reason for using the value 2, but it provided the best results in > > terms of performance without any noticeable impact. > > > > mm/slub.c | 17 +++++++---------- > > 1 file changed, 7 insertions(+), 10 deletions(-) > > > > diff --git a/mm/slub.c b/mm/slub.c > > index c87628cd8a9a..8f6f38083b94 100644 > > --- a/mm/slub.c > > +++ b/mm/slub.c > > @@ -287,6 +287,7 @@ static inline bool > > kmem_cache_has_cpu_partial(struct kmem_cache *s) > > #define OO_SHIFT 16 > > #define OO_MASK ((1 << OO_SHIFT) - 1) > > #define MAX_OBJS_PER_PAGE 32767 /* since slab.objects is u15 > > */ > > +#define SLUB_PAGE_FRAC_SHIFT 12 > > > > /* Internal SLUB flags */ > > /* Poison object */ > > @@ -4117,6 +4118,7 @@ static inline int calculate_order(unsigned > > int size) > > unsigned int min_objects; > > unsigned int max_objects; > > unsigned int nr_cpus; > > + unsigned int page_size_frac; > > > > /* > > * Attempt to find best configuration for a slab. This > > @@ -4145,10 +4147,13 @@ static inline int calculate_order(unsigned > > int size) > > max_objects = order_objects(slub_max_order, size); > > min_objects = min(min_objects, max_objects); > > > > - while (min_objects > 1) { > > + page_size_frac = ((PAGE_SIZE >> SLUB_PAGE_FRAC_SHIFT) == 1) > > ? 0 > > + : PAGE_SIZE >> SLUB_PAGE_FRAC_SHIFT; > > + > > + while (min_objects >= 1) { > > unsigned int fraction; > > > > - fraction = 16; > > + fraction = 16 + page_size_frac; > > while (fraction >= 4) { > > Sorry I'm a bit late for the review. > > IIRC hexagon/powerpc can have ridiculously large page sizes (1M or > 256KB) > (but I don't know if such config is actually used, tbh) so I think > there should be > an upper bound. Hi, I think that might not be required as arch with larger page size will required larger fraction value as per this exit condition (rem <= slab_size / fract_leftover) during calc_slab_order. > > > order = calc_slab_order(size, min_objects, > > slub_max_order, fraction); > > @@ -4159,14 +4164,6 @@ static inline int calculate_order(unsigned > > int size) > > min_objects--; > > } > > - /* > > - * We were unable to place multiple objects in a slab. Now > > - * lets see if we can place a single object there. > > - */ > > - order = calc_slab_order(size, 1, slub_max_order, 1); > > - if (order <= slub_max_order) > > - return order; > > I'm not sure if it's okay to remove this? > It was fine in v2 because the least wasteful order was chosen > regardless of fraction but that's not true anymore. > Ok, So my though are like if single object in slab with slab_size = PAGE_SIZE << slub_max_order and it wastage more then 1\4th of slab_size then it's better to skip this part and use MAX_ORDER instead of slub_max_order. Could you kindly share your perspective on this part? Tha nks Jay Patel > Otherwise, everything looks fine to me. I'm too dumb to anticipate > the outcome of increasing the slab order :P but this patch does not > sound crazy to me. > > Thanks! > -- > Hyeonggon