From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64094EB64DA for ; Thu, 20 Jul 2023 10:31:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA2BA2800EA; Thu, 20 Jul 2023 06:31:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A52BE28004C; Thu, 20 Jul 2023 06:31:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CCFC2800EA; Thu, 20 Jul 2023 06:31:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 78D3C28004C for ; Thu, 20 Jul 2023 06:31:12 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 477391202D3 for ; Thu, 20 Jul 2023 10:31:12 +0000 (UTC) X-FDA: 81031622784.03.2E319EB Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf25.hostedemail.com (Postfix) with ESMTP id B4803A0011 for ; Thu, 20 Jul 2023 10:31:09 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=fqpEzH+t; spf=pass (imf25.hostedemail.com: domain of jaypatel@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=jaypatel@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689849069; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ylbuyxAHrNAR8XD1H0ZqkUsrji8uiaAMk7DNXmmJJ+E=; b=2nAtT0G2jU6FToNc4OuM6hNTyxqt02469a9cBD2axgQCTvm88KJSxPNbItqGXHA5LCimRJ oU8W49DStdZ7YBEIhgozYv6Fb0exqjxLPf3/mEvSMLpPt0z+MNx8SS1KKr8ZeNBaI9PI7B /1FvWfWshcHjwkWyzBOB0oRNAuFXI7Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689849069; a=rsa-sha256; cv=none; b=IND3KnQd9Louiw+x6O2LpZnUCVPsibEBbZ7AKbCjGyApwxmteMwb0S1EVzoDW1YD07LKGV ssKbWfSBs0xdN/R5/TJeLmT3aWjUoVTdeZVAL0eehW0E+6Jx237t5ZAiYqSVgzq4w1X/Mn ayGB2RbJoLCg8XKkROmHviUsRcARg84= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=fqpEzH+t; spf=pass (imf25.hostedemail.com: domain of jaypatel@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=jaypatel@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com Received: from pps.filterd (m0353724.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36K9cg5N001505; Thu, 20 Jul 2023 10:31:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : reply-to : to : cc : date : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=pp1; bh=ylbuyxAHrNAR8XD1H0ZqkUsrji8uiaAMk7DNXmmJJ+E=; b=fqpEzH+tfgSVsOwjSUzzzBW3qosrML2Pa9duWDhGSWm2ChtfOQeD86r6i4ftHdDanFBg hBVU7Cceks33MSbTbZFs9dAJUNcJUm5aMpx6InA06IvQKslCLjzVW64QzwkEqdIBfUAj lQUrdCX7WXpw9WsqWt+giVgWeeQQIe9lQDO72+Td5fVCi4HtwNx9MXeTvIcKB82Aqbd6 3R5FURSLb0ptsCH2ronhQnCMbA9JVQTnRadhKrhTMmd+vI71wpeQM2pdwdhDzzqlQBKO Aqud4Aw7uenyStfNh6/0gkl2Q/bVGZ644WU5/znLymvmzu/sBSLu3jIuPjHQEfJqdSLr JA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rxqs3pwpp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 20 Jul 2023 10:31:05 +0000 Received: from m0353724.ppops.net (m0353724.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 36KA3LDe001270; Thu, 20 Jul 2023 10:31:04 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rxqs3pwph-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 20 Jul 2023 10:31:04 +0000 Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 36K9frP5029098; Thu, 20 Jul 2023 10:31:04 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([172.16.1.70]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3rv6smp5ny-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 20 Jul 2023 10:31:04 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 36KAV3Iu2425524 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 20 Jul 2023 10:31:03 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F9E058072; Thu, 20 Jul 2023 10:31:03 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A6FC15806C; Thu, 20 Jul 2023 10:30:58 +0000 (GMT) Received: from patel (unknown [9.43.75.201]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Thu, 20 Jul 2023 10:30:58 +0000 (GMT) Message-ID: Subject: Re: [PATCH] [RFC PATCH v2]mm/slub: Optimize slub memory usage From: Jay Patel Reply-To: jaypatel@linux.ibm.com To: Vlastimil Babka , linux-mm@kvack.org Cc: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, tsahu@linux.ibm.com, piyushs@linux.ibm.com Date: Thu, 20 Jul 2023 16:00:56 +0530 In-Reply-To: References: <20230628095740.589893-1-jaypatel@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 (3.28.5-22.el8) X-TM-AS-GCONF: 00 X-Proofpoint-GUID: UFOj_LLkYsTx08rIECIwpGV2cazyIFQU X-Proofpoint-ORIG-GUID: CuKhdxbnfLrmyDXGzCc_vyXWDcy8gqj_ Content-Transfer-Encoding: 7bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-20_04,2023-07-19_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 spamscore=0 mlxscore=0 clxscore=1015 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2307200084 X-Rspamd-Queue-Id: B4803A0011 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: d9mn3pma8jw7ynzho4mx1ynxjntbpngp X-HE-Tag: 1689849069-521726 X-HE-Meta: U2FsdGVkX19KycjLVbm55E+6XH7TFkjSfh4RtCnGovDpUk4tZ2Sc3Jvd+419DKmz3mcV5Jmn+cZ0YpGjyqheGP1IwkPMviFfR+aYmZYWlkrKjZdtwlz6SCnicaYZqbWIlVwoUTTYGKMuydqK4S4r/7nXL/zl1j5b49QLRRZynE1SQZNiZ6q6hsKFoSvtKtyCBaKH1++frW3ibwMD2Q9mOotMDf1sqbQhbtZgx0XeScPHI+hTj+H69v/TLom1KA/a/+RDxKfqbH7+bQzcWWCCShjr+jOLF6T3kQlnoTwBZPMDW4P0QPf6pfKun3nO1ZxIkbG5Er2I95EpUzEeBkKuzS8A74v1aJ4gjsNEFaSrnq+WdHKWj67jWjsvugAEs0hRMyKyMrs608GX5d56arCUmwgvqhVQlGRrETQi6PzFa2eMwg+TarQauPL8fCgv6ba07P3/o27LC0CLaCI0M3tKQMHF4K9AU2w09tKdae8+CUK6M9lBcEKtmgOQirouDZ61nwY0AfOSjWonQsDqdDph3lF5Gf/HCf8U+WtSEidKOWHusbCuanqwltYWSdNasydegVgtFelvHIYcO6sGZH++pYAEIQYurqBhVyNwxZT32BGloIba/CLCFHT44QEHVR02KZbNab70EiBpzWWJ8vTKqHX3GdaUezisarLbPeThhughkzeNkGPgZqKZGdDGPHwXdC4WAfpmzGvHfnTNBnF+TfehEam9ybcTKhpk8gXB1tQujfU30d3XzQ02XwpAXZynV7+B0r79ZdITmLkOFvxesDfyVJGwlrc9FA5HAkGXgXVvVxViR4KAFw+NazP/wlHLvpHa6RxpskAZWIxdlWp8Ce6W0LHUV9aBehdvJFEIPnbNp7U4WwFG+c0x0eIf2dHvkqkrK+pXf3Iq6JSOegVaKEiuO29Tkm5Ua0epR5mKtalmZ/hu4PmD/lZmfM2jFb43hy9u8K4SeD2UkSv3plc rkeiOB8X id5DrrlzDhP2ZiO5J/9sc+fj2bl5n21nnRmvrLR1aqhm2qnDP4MxZdREy9aUmy7tTQbZR+aKHdHuNZ7e6oSiVZPKXyPTQHiiu7u03ZLoQYDZOPnJYY5CuJcA74VXVL7D6MOqN4o7pKT/vQk4o1ZBf8GRmYoWUYE9iXdO0fSMUK5xcUVEIw1vnTSdFeVy0PqG7cQId3Iz7xpoddqYzmYAkkSct9uaTrzi0HLvGJbSkIx5Q/pZFhzrjR23v5orduWOdl60TNxpZlCZKmpMI05iEmI+nyh4BTpe/3yR6peQ5G9z8BnzWWmRxUDDD2iSX70C81gMN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 2023-07-12 at 15:06 +0200, Vlastimil Babka wrote: > On 6/28/23 11:57, Jay Patel wrote: > > In the previous version [1], we were able to reduce slub memory > > wastage, but the total memory was also increasing so to solve > > this problem have modified the patch as follow: > > > > 1) If min_objects * object_size > PAGE_ALLOC_COSTLY_ORDER, then it > > will return with PAGE_ALLOC_COSTLY_ORDER. > > 2) Similarly, if min_objects * object_size < PAGE_SIZE, then it > > will > > return with slub_min_order. > > 3) Additionally, I changed slub_max_order to 2. There is no > > specific > > reason for using the value 2, but it provided the best results in > > terms of performance without any noticeable impact. > > > > [1] > > > > Hi, > > thanks for the v2. A process note: the changelog should be self- > contained as > will become the commit description in git log. What this would mean > here is > to take the v1 changelog and adjust description to how v2 is > implemented, > and of course replace the v1 measurements with new ones. > > The "what changed since v1" can be summarized in the area after sign- > off and > "---", before the diffstat. This helps those that looked at v1 > previously, > but doesn't become part of git log. > > Now, my impression is that v1 made a sensible tradeoff for 4K pages, > as the > wastage was reduced, yet overal slab consumption didn't increase > much. But > for 64K the tradeoff looked rather bad. I think it's because with 64K > pages > and certain object size you can e.g. get less waste with order-3 than > order-2, but the difference will be relatively tiny part of the 64KB, > so > it's not worth the increase of order, while with 4KB you can get > larger > reduction of waste both in absolute amount and especially relatively > to the > 4KB size. > > So I think ideally the calculation would somehow take this into > account. The > changes done in v2 as described above are different. It seems as a > result we > can now calculate lower orders on 4K systems than before the patch, > probably > due to conditions 2) or 3) ? I think it would be best if the patch > resulted > only in the same or higher order. It should be enough to tweak some > thresholds for when it makes sense to pay the price of higher order - > whether the reduction of wastage is worth it, in a way that takes the > page > size into account. > > Thanks, > Vlastimil Hi Vlastimil, Indeed, I aim to optimize memory allocation in the SLUB allocator [1] by targeting larger page sizes with minimal modifications , resulting in reduced memory consumpion. [1]https://lore.kernel.org/linux-mm/20230720102337.2069722-1- jaypatel@linux.ibm.com/ Thanks, Jay Patel > > > I have conducted tests on systems with 160 CPUs and 16 CPUs using > > 4K > > and 64K page sizes. The tests showed that the patch successfully > > reduces the total and wastage of slab memory without any noticeable > > performance degradation in the hackbench test. > > > > Test Results are as follows: > > 1) On 160 CPUs with 4K Page size > > > > +----------------+----------------+----------------+ > > > Total wastage in slub memory | > > +----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 2090 Kb | 3204 Kb | > > > With Patch | 1825 Kb | 3088 Kb | > > > Wastage reduce | ~12% | ~4% | > > +----------------+----------------+----------------+ > > > > +-----------------+----------------+----------------+ > > > Total slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 500572 | 713568 | > > > With Patch | 482036 | 688312 | > > > Memory reduce | ~4% | ~3% | > > +-----------------+----------------+----------------+ > > > > hackbench-process-sockets > > +-------+-----+----------+----------+-----------+ > > > | Normal |With Patch| | > > +-------+-----+----------+----------+-----------+ > > > Amean | 1 | 1.3237 | 1.2737 | ( 3.78%) | > > > Amean | 4 | 1.5923 | 1.6023 | ( -0.63%) | > > > Amean | 7 | 2.3727 | 2.4260 | ( -2.25%) | > > > Amean | 12 | 3.9813 | 4.1290 | ( -3.71%) | > > > Amean | 21 | 6.9680 | 7.0630 | ( -1.36%) | > > > Amean | 30 | 10.1480 | 10.2170 | ( -0.68%) | > > > Amean | 48 | 16.7793 | 16.8780 | ( -0.59%) | > > > Amean | 79 | 28.9537 | 28.8187 | ( 0.47%) | > > > Amean | 110 | 39.5507 | 40.0157 | ( -1.18%) | > > > Amean | 141 | 51.5670 | 51.8200 | ( -0.49%) | > > > Amean | 172 | 62.8710 | 63.2540 | ( -0.61%) | > > > Amean | 203 | 74.6417 | 75.2520 | ( -0.82%) | > > > Amean | 234 | 86.0853 | 86.5653 | ( -0.56%) | > > > Amean | 265 | 97.9203 | 98.4617 | ( -0.55%) | > > > Amean | 296 | 108.6243 | 109.8770 | ( -1.15%) | > > +-------+-----+----------+----------+-----------+ > > > > 2) On 160 CPUs with 64K Page size > > +-----------------+----------------+----------------+ > > > Total wastage in slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot |After Hackbench | > > > Normal | 919 Kb | 1880 Kb | > > > With Patch | 807 Kb | 1684 Kb | > > > Wastage reduce | ~12% | ~10% | > > +-----------------+----------------+----------------+ > > > > +-----------------+----------------+----------------+ > > > Total slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 1862592 | 3023744 | > > > With Patch | 1644416 | 2675776 | > > > Memory reduce | ~12% | ~11% | > > +-----------------+----------------+----------------+ > > > > hackbench-process-sockets > > +-------+-----+----------+----------+-----------+ > > > | Normal |With Patch| | > > +-------+-----+----------+----------+-----------+ > > > Amean | 1 | 1.2547 | 1.2677 | ( -1.04%) | > > > Amean | 4 | 1.5523 | 1.5783 | ( -1.67%) | > > > Amean | 7 | 2.4157 | 2.3883 | ( 1.13%) | > > > Amean | 12 | 3.9807 | 3.9793 | ( 0.03%) | > > > Amean | 21 | 6.9687 | 6.9703 | ( -0.02%) | > > > Amean | 30 | 10.1403 | 10.1297 | ( 0.11%) | > > > Amean | 48 | 16.7477 | 16.6893 | ( 0.35%) | > > > Amean | 79 | 27.9510 | 28.0463 | ( -0.34%) | > > > Amean | 110 | 39.6833 | 39.5687 | ( 0.29%) | > > > Amean | 141 | 51.5673 | 51.4477 | ( 0.23%) | > > > Amean | 172 | 62.9643 | 63.1647 | ( -0.32%) | > > > Amean | 203 | 74.6220 | 73.7900 | ( 1.11%) | > > > Amean | 234 | 85.1783 | 85.3420 | ( -0.19%) | > > > Amean | 265 | 96.6627 | 96.7903 | ( -0.13%) | > > > Amean | 296 | 108.2543 | 108.2253 | ( 0.03%) | > > +-------+-----+----------+----------+-----------+ > > > > 3) On 16 CPUs with 4K Page size > > +-----------------+----------------+------------------+ > > > Total wastage in slub memory | > > +-----------------+----------------+------------------+ > > > | After Boot | After Hackbench | > > > Normal | 491 Kb | 727 Kb | > > > With Patch | 483 Kb | 670 Kb | > > > Wastage reduce | ~1% | ~8% | > > +-----------------+----------------+------------------+ > > > > +-----------------+----------------+----------------+ > > > Total slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 105340 | 153116 | > > > With Patch | 103620 | 147412 | > > > Memory reduce | ~1.6% | ~4% | > > +-----------------+----------------+----------------+ > > > > hackbench-process-sockets > > +-------+-----+----------+----------+---------+ > > > | Normal |With Patch| | > > +-------+-----+----------+----------+---------+ > > > Amean | 1 | 1.0963 | 1.1070 | ( -0.97%) | > > > Amean | 4 | 3.7963) | 3.7957 | ( 0.02%) | > > > Amean | 7 | 6.5947) | 6.6017 | ( -0.11%) | > > > Amean | 12 | 11.1993) | 11.1730 | ( 0.24%) | > > > Amean | 21 | 19.4097) | 19.3647 | ( 0.23%) | > > > Amean | 30 | 27.7023) | 27.6040 | ( 0.35%) | > > > Amean | 48 | 44.1287) | 43.9630 | ( 0.38%) | > > > Amean | 64 | 58.8147) | 58.5753 | ( 0.41%) | > > +-------+----+---------+----------+-----------+ > > > > 4) On 16 CPUs with 64K Page size > > +----------------+----------------+----------------+ > > > Total wastage in slub memory | > > +----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 194 Kb | 349 Kb | > > > With Patch | 191 Kb | 344 Kb | > > > Wastage reduce | ~1% | ~1% | > > +----------------+----------------+----------------+ > > > > +-----------------+----------------+----------------+ > > > Total slub memory | > > +-----------------+----------------+----------------+ > > > | After Boot | After Hackbench| > > > Normal | 330304 | 472960 | > > > With Patch | 319808 | 458944 | > > > Memory reduce | ~3% | ~3% | > > +-----------------+----------------+----------------+ > > > > hackbench-process-sockets > > +-------+-----+----------+----------+---------+ > > > | Normal |With Patch| | > > +-------+----+----------+----------+----------+ > > > Amean | 1 | 1.9030 | 1.8967 | ( 0.33%) | > > > Amean | 4 | 7.2117 | 7.1283 | ( 1.16%) | > > > Amean | 7 | 12.5247 | 12.3460 | ( 1.43%) | > > > Amean | 12 | 21.7157 | 21.4753 | ( 1.11%) | > > > Amean | 21 | 38.2693 | 37.6670 | ( 1.57%) | > > > Amean | 30 | 54.5930 | 53.8657 | ( 1.33%) | > > > Amean | 48 | 87.6700 | 86.3690 | ( 1.48%) | > > > Amean | 64 | 117.1227 | 115.4893 | ( 1.39%) | > > +-------+----+----------+----------+----------+ > > > > Signed-off-by: Jay Patel > > --- > > mm/slub.c | 52 +++++++++++++++++++++++++------------------------ > > --- > > 1 file changed, 25 insertions(+), 27 deletions(-) > > > > diff --git a/mm/slub.c b/mm/slub.c > > index c87628cd8a9a..0a1090c528da 100644 > > --- a/mm/slub.c > > +++ b/mm/slub.c > > @@ -4058,7 +4058,7 @@ EXPORT_SYMBOL(kmem_cache_alloc_bulk); > > */ > > static unsigned int slub_min_order; > > static unsigned int slub_max_order = > > - IS_ENABLED(CONFIG_SLUB_TINY) ? 1 : PAGE_ALLOC_COSTLY_ORDER; > > + IS_ENABLED(CONFIG_SLUB_TINY) ? 1 : 2; > > static unsigned int slub_min_objects; > > > > /* > > @@ -4087,11 +4087,10 @@ static unsigned int slub_min_objects; > > * the smallest order which will fit the object. > > */ > > static inline unsigned int calc_slab_order(unsigned int size, > > - unsigned int min_objects, unsigned int max_order, > > - unsigned int fract_leftover) > > + unsigned int min_objects, unsigned int max_order) > > { > > unsigned int min_order = slub_min_order; > > - unsigned int order; > > + unsigned int order, min_wastage = size, min_wastage_order = > > MAX_ORDER+1; > > > > if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE) > > return get_order(size * MAX_OBJS_PER_PAGE) - 1; > > @@ -4104,11 +4103,17 @@ static inline unsigned int > > calc_slab_order(unsigned int size, > > > > rem = slab_size % size; > > > > - if (rem <= slab_size / fract_leftover) > > - break; > > + if (rem < min_wastage) { > > + min_wastage = rem; > > + min_wastage_order = order; > > + } > > } > > > > - return order; > > + if (min_wastage_order <= slub_max_order) > > + return min_wastage_order; > > + else > > + return order; > > + > > } > > > > static inline int calculate_order(unsigned int size) > > @@ -4142,35 +4147,28 @@ static inline int calculate_order(unsigned > > int size) > > nr_cpus = nr_cpu_ids; > > min_objects = 4 * (fls(nr_cpus) + 1); > > } > > + > > + if ((min_objects * size) > (PAGE_SIZE << > > PAGE_ALLOC_COSTLY_ORDER)) > > + return PAGE_ALLOC_COSTLY_ORDER; > > + > > + if ((min_objects * size) <= PAGE_SIZE) > > + return slub_min_order; > > + > > max_objects = order_objects(slub_max_order, size); > > min_objects = min(min_objects, max_objects); > > > > - while (min_objects > 1) { > > - unsigned int fraction; > > - > > - fraction = 16; > > - while (fraction >= 4) { > > - order = calc_slab_order(size, min_objects, > > - slub_max_order, fraction); > > - if (order <= slub_max_order) > > - return order; > > - fraction /= 2; > > - } > > + while (min_objects >= 1) { > > + order = calc_slab_order(size, min_objects, > > + slub_max_order); > > + if (order <= slub_max_order) > > + return order; > > min_objects--; > > } > > > > - /* > > - * We were unable to place multiple objects in a slab. Now > > - * lets see if we can place a single object there. > > - */ > > - order = calc_slab_order(size, 1, slub_max_order, 1); > > - if (order <= slub_max_order) > > - return order; > > - > > /* > > * Doh this slab cannot be placed using slub_max_order. > > */ > > - order = calc_slab_order(size, 1, MAX_ORDER, 1); > > + order = calc_slab_order(size, 1, MAX_ORDER); > > if (order <= MAX_ORDER) > > return order; > > return -ENOSYS;