From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 916E73A267 for ; Wed, 20 Mar 2024 14:11:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710943874; cv=fail; b=E6jB8VUCygqwt46m/oWwAusBJBP5OtWxv5Jg5CjXb3fbpXO3wP4Z9nktuDGWy9dr8rexuyY/+dpvA6jPH3BqBWXGUIDMIGPm+5Mj05KO9IPmNGFTMX2wQ6ArRx2AFqW5VbPEbtTrR5hKLj0ruV3RQ6Tsb+giO8+yS0GgJuoKs5Y= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710943874; c=relaxed/simple; bh=ROndGG9MmE91ha/+6BqL2zSUMJxmlpZNLJAlz0ZgOVU=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=AJiNOONF0H4Ad2oeE0a64hhmDVylXhsABA/h2dyheRWs8LjBAqAlzNYuJf4P3GGx58R0yAWNJlpsWWShM4fdWHl+O+Q7pb1zrU9mYewr3VHQ0VHuOgRT2AyH3EItUMnP5+gFvj7kvRduXuB66OrC2BVNipIr6bYqb/e4ILrz0Uo= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=OFIoAQb9; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=CymxFc8h; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="OFIoAQb9"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="CymxFc8h" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 42KBwoeE028579; Wed, 20 Mar 2024 14:10:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : content-type : in-reply-to : mime-version; s=corp-2023-11-20; bh=qyRy0jADkfoGrjIjufpkhdkK3f+L4BL3kLTOynzlYeg=; b=OFIoAQb9oa07iRLb7Tiu/tJoTBGhWdHwArItHH37n1o7v1GoV+jGh/hYPRPuyQUrEg33 LHhhz/wAmmeSee+ND2nst54pxOBM6fFVDPDI2BwHd+B6EzUnjAdrmnSHoBHWjsb2QqRP +jOjbEarFxsXLsQjROnjWErEkoyk6ft3Olu7ajxqcJ++CXqcSqlL9tTM7Er5qAJ1i6uT bhh9T+poTOkTFmy9EO4aVHZtDZsZsjZetn+9aAvjqPUROyd0SF29lfitZOh1bV5iI2Hx f3BizQylXw+WJJ+JYHPDGaWbhPooU+CmHqX6ppjlx3d9kalTA9BhXhz7JAjOF5sjzDND Vg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ww1udg546-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2024 14:10:54 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 42KDvHqC007472; Wed, 20 Mar 2024 14:10:53 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2168.outbound.protection.outlook.com [104.47.58.168]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3ww1v8d8ws-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Mar 2024 14:10:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GPPn4EN9Gosled7fpu09B2srSZFiS+GZ2N1Tt9qqJjaHO9kXEEy/+nUmVN3PmtJ0EKp77VkysqJVkalxGukUvWAz0H8RtBFADvFGNH3Cy4dtaSbscYbYvnWE4Mix3FhaqenI55p2ScWmJdJSeA0oruXVXP7rmBYof9IJtVt6YE5bYKvEfZMql4hj+7juMwIqFzcFgE51zKnrGjZSZFW0FcHoGqbtKYJHvC3PoJvA2CJfHmA280kjs6UjjUY6PbZl8gIs1RCp6CKs1Hzsx31hF97WHQrNVvYJkLKgUOoWK7IOh/GyNFU5nY5HshTwSl4H+oA1bwvm+k5BzBkgCcnhyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qyRy0jADkfoGrjIjufpkhdkK3f+L4BL3kLTOynzlYeg=; b=JwtEe1MikcpRlQue3g6ztShfJeOELmMQNDfGVouJ8Yin5L2PmTSM171T5hNKT+fxsoTurHXKxDSI5mEHz1t8H7FFh7GdA+d0/EjNnLk4Z2ryEtZs+yD5zHJr5mCeTxwDLE9JyuIqEI4YSN91emLRCecugh7k0bWMg6FdSnH7g5OL/fOgaIIg005Bi7QavSzR+4NMU0DjgIAyGRxHFZI/2sYGc4ZI1SGaRIpaR6DhTjQHH/TkBhZQzGwJ7zuRcQIG4q9kb46FtKVv7+WhQqaa6K5w1HRg7ZF2/FtQYXgjrUlFqKj7EZZPNzfDN/5Oz/dvluPMX4wCse2EuHiiPsKI2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qyRy0jADkfoGrjIjufpkhdkK3f+L4BL3kLTOynzlYeg=; b=CymxFc8hxola9A6dvt8Jmhy4Uh7sXAMVmYHnjxcobHnBJtHGv+p3sLMr8ifzNhbYXuCzHf7k9f0XpjKFJu4004F7HKqGzDJhhs8AnqA+eBiDRO0sS5+kRAfprT1o5O6VJAQJX5lzAOhkUwLkdHFY9uljM7HnuHtALPHKv7Co70Y= Received: from BN0PR10MB5128.namprd10.prod.outlook.com (2603:10b6:408:117::24) by SA1PR10MB6391.namprd10.prod.outlook.com (2603:10b6:806:257::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.31; Wed, 20 Mar 2024 14:10:51 +0000 Received: from BN0PR10MB5128.namprd10.prod.outlook.com ([fe80::ad12:a809:d789:a25b]) by BN0PR10MB5128.namprd10.prod.outlook.com ([fe80::ad12:a809:d789:a25b%4]) with mapi id 15.20.7386.031; Wed, 20 Mar 2024 14:10:51 +0000 Date: Wed, 20 Mar 2024 10:10:43 -0400 From: Chuck Lever To: Jeff Layton Cc: kdevops@lists.linux.dev Subject: Re: [PATCH 8/8] nfsd: default to a more sane number of threads Message-ID: References: <20240320-fixes-v1-0-d1567776ad50@kernel.org> <20240320-fixes-v1-8-d1567776ad50@kernel.org> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240320-fixes-v1-8-d1567776ad50@kernel.org> X-ClientProxiedBy: CH5P220CA0006.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:1ef::29) To BN0PR10MB5128.namprd10.prod.outlook.com (2603:10b6:408:117::24) Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN0PR10MB5128:EE_|SA1PR10MB6391:EE_ X-MS-Office365-Filtering-Correlation-Id: fc15da22-47cf-4c79-73c7-08dc48e78c9a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9i63tWmD5SjTzUznXPMQmi6xIBsqUn3jQFKrnNi2gsOTYmAT6bg4wy02OqWcxYjQR5c6nrzhs9RozAXpfTaKoKTGinWMCYUpr+6Xf2JlnP9tRJdV9TuLHstcrxJVQ5WCLeHj1TmU/5jEyzzAgAtfFBaVRBHNfnjzXVbY1pfwPQNRsBhbQ6AwmkopOyrx82qrS1u2CRyt+roeLrmMAx3MY0giYbGr44gomPT2ugrx/2ZWUMpSsLBRMe2YV2O7+UsAP4tGbosQb5e1h4g6qdpV5par4wiZfJ/QKF/qBU95SOFZPlDSl86j0fhTGdIQVmgMoUTtMdq41rArAeZRv/5nnLOG9qA+oo36z84AHCsrcEfudp0nVpdz5Omr7hvDrU71pDe8w7/jFzX0SczqpqGTS247MEPSFyK4sqWtZRj4buOji9MDkHS8t/1jxu9tyx4Fi2Zi271QFn0T/U0kZ3q/PYoTmrs2Lg63tY3AvmkzBNtPV+5P3bBkSehAbfAwAEpf3QBmSLvzY2Gk8wg1ez0ZGVHDNowImkHlhBKc2jJ9VU5+6AEcGmZRIl62JcwEK+AoXxTiXdaeyck5Fid84KPyAzA42/QGjbjyky8WUL3HUuTHtY0udM6NeodImRGho/hdxFEnPacewAPVlJX6Xxzla71PRLI/lMmSkZdMXdUdDaQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN0PR10MB5128.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?s+aO/XwvWfpHLA+fm522+Ul1xaiqq/1orLLGwVkBiOqpwBTTQL3kOX76w+0n?= =?us-ascii?Q?JFlgget+O4g/jb0P1Ch/ItGooILQgG2q2FJXjg0MXrQPwYXbw8sIss60QdrP?= =?us-ascii?Q?vQk1WTDpjRSXuxzLkf49OHPvwJgGzY1+RJ5/qh3htKr617ZnPWZY0E1cXi7g?= =?us-ascii?Q?j9UZRRfCBk5KelpAVAYDx19MYxs2V6tS/9Ni/LCc5cL48ktJC03EHb5mZQSW?= =?us-ascii?Q?EjC7raMUhJzbgZ5/d0Cc/I7DNTUWwNTky5i2q3IujJJ4Rtod3eDpuDVi5sJ2?= =?us-ascii?Q?PLzerg/g7EYoTaOKsudldnbnlAIWzwvTKasyQdnYi2OPmFbEYuMLsfQc0Vb1?= =?us-ascii?Q?wSJMEhhUxJTVfkzTbJNFuBOJF/xqwueEOGHZVW3fsmE1M2CsXni4ekJsEfTp?= =?us-ascii?Q?EcJ6SpN+UKGrGL33/NcuDvYXjM0kKK4rfwltvDAPAEV+mic9AK90ZKE9DTxi?= =?us-ascii?Q?lDrU0lZUbXgUcTUVBF+vsedkSFcOF9sU0Ezm0jn8KTtui1w/bhu21Qc9UHRe?= =?us-ascii?Q?7sv3F4HupzDpHYzdHNdePDK045q+oD8e3hOqvyh2eItmvD54zi+p1qSftGGz?= =?us-ascii?Q?8GbOcWbhAQonz8KHFXeoRp6NdVWAT7Mvl7abIMlTTY6PFKipJuD7GuAKz+4J?= =?us-ascii?Q?DfUna/vtSPQwFDMbJgGdw5HI2M/Y6u7K77DKRqPWOtZwKg0hdqmqVVGQ36R3?= =?us-ascii?Q?Iu8FeI5zeRlkLNU1lQqd8wd4CYP9oYNytFXIDQaUudz1GMppVY7w2VHGEtCV?= =?us-ascii?Q?scawCRReQT8A/W3hznRXiYgOcG8AYCSaw3vSRprGxzj2nFGPP9xM4w6Vg0Z1?= =?us-ascii?Q?OYFkijQpCVISqGnXO9Dcu+h/GFbRjJxgzUXVG5paLWWJfxAjm6eLeLfnIeka?= =?us-ascii?Q?3xPQeSAvfe1JZM9Zjk7lLdy5YW2jnVVtjnclRC7PTWykHeO7N+C8Ru3B5I4Y?= =?us-ascii?Q?n+LiGv4qLRQYA8GeudTZUbZcwvEpIMZAfIW0W30T4toBbVLpwN9EK2/jHcGm?= =?us-ascii?Q?o2/XXNSOdD05f7TPK2z6lh1WrZQblP3WZdOla4ahcSyBjsqUoY0wVRqVBktw?= =?us-ascii?Q?0egrcs98RzPe4zDags+Gon8W6DAn0b+4UMO6ZuIlvzEp6/A1r46Y1cbxuJuW?= =?us-ascii?Q?TEnIYI9Pk3MGHHO7XC3RQW1oagao7rUakmYafUp1IxmMj/ZqTZLD/gvQO/VX?= =?us-ascii?Q?N9LOzQdqVog1walevHiEXf/3UrYM+wM4/Q6BuBPQ26p9issvJ+qmzTlG2h9d?= =?us-ascii?Q?GvFcmdHKK5z7RpORiy0udL7oegB679dy6QsB0QAcc/wAYTrrAFHMBK2K4iqW?= =?us-ascii?Q?Yw7PFcCYjC4oZhs3R8S3106FlqAjUkTl+cqWK7EqikJaDsU4VjOKtQabfJ/h?= =?us-ascii?Q?wt3WI6VdGYl0v0BLMHtI8US5BvK9uheATS0rpGUXUaP2+rS2u2QhN/ogxk6I?= =?us-ascii?Q?Rcyh4LWqaPAQzIUEyrNgPr7yzznJ4Z8mq2FyWxtE6ESEXDgOJHI+PMl8rXep?= =?us-ascii?Q?Dgyb7paQLtkccdI7hHQlhSBQUmECKeOBg034irR8U55bnDNVgIWTHBycLAvC?= =?us-ascii?Q?6k494c0UMiSvF+mm8kFQYT4foBm8ud6WSRkrriqLIjvnFIRssdhKAuBRRslL?= =?us-ascii?Q?UA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: y/cWcTqPzCV5xF9PF06T5P1k21H6rVvhHPBVEi6wCM/4MInk5GGuI7wkiJh0CCL2sx39fPFVpSPQFdNCPVkZkwMqx3/8y6Ck8jks3hlfowIXUZ5eD/I0ue018Xwd0bAuM2B2xn/T5DFYOSwAGpzQXOM+Ekg2aw5Mei3q//Y/PDU4Jqw0BXrZodeL5IBJaBUm6DiM6jRDmPHf3sjdfqLMAbQl2Xfac8b3896XfeJOTLl4CM02YT0XyItwEYnHITVlcCuzuF38dnC5m3PxOZ9EKu8vDq1oL6ukrQJu/tOxWzoYSCTmjM86Fnik2oe7dOQQ5qc6exiiQGPsj4xwBiKchGZ7Y8m8df9Ae1PTVHaFh167kW9bfn/3Xsi8w9FyDpu1ZfMzESgQcRCtVmMOs8/G7hA1ve8DKSAcJbGcacuRrG7njpuGOpUnzg1YiYFxSYuT9HFbF110Tx8gyx5dAbehcp0e8ZykyJ0elB6hiBO3vI5U2IYfWtXU36fAnhr7dQwIWO0olrOUbLGLZ667wY5HqogtXu96o7L0qDpNWbC5t6KOvh4HG2dYimA4oYrN9Yk0+VvjhAB1TqGiZMnpt5invCJOvZzm5SbhccYFGRoI958= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: fc15da22-47cf-4c79-73c7-08dc48e78c9a X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2024 14:10:51.4251 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: c1t16p45vz5yEh8zwSrZyHqlM0zlqxwIz201yhI6JlhrjdOz7uU/Ub1TPiW4YQQviypenodxJT8xruQhKBPIRw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB6391 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-20_09,2024-03-18_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 mlxscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2403140000 definitions=main-2403200112 X-Proofpoint-GUID: f_aQfJx7U2LDe6P2rRUjoWU3fONOwrs5 X-Proofpoint-ORIG-GUID: f_aQfJx7U2LDe6P2rRUjoWU3fONOwrs5 On Wed, Mar 20, 2024 at 09:11:31AM -0400, Jeff Layton wrote: > Add a small script that can estimate the number of threads for > nfsd. We default to 32 threads for every GB of guest memory. > > Signed-off-by: Jeff Layton > --- > kconfigs/Kconfig.nfsd | 2 +- > scripts/nfsd-thread-est.sh | 15 +++++++++++++++ > 2 files changed, 16 insertions(+), 1 deletion(-) > > diff --git a/kconfigs/Kconfig.nfsd b/kconfigs/Kconfig.nfsd > index dec98c1e964f..7c28ad98955a 100644 > --- a/kconfigs/Kconfig.nfsd > +++ b/kconfigs/Kconfig.nfsd > @@ -69,7 +69,7 @@ config NFSD_EXPORT_OPTIONS > > config NFSD_THREADS > int "Number of nfsd threads to spawn" > - default 8 > + default $(shell, scripts/nfsd-thread-est.sh) > help > Number of nfsd threads to start up for testing. > > diff --git a/scripts/nfsd-thread-est.sh b/scripts/nfsd-thread-est.sh > new file mode 100755 > index 000000000000..dc5a1deb1215 > --- /dev/null > +++ b/scripts/nfsd-thread-est.sh > @@ -0,0 +1,15 @@ > +#!/bin/bash > +# > +# The default number of 8 nfsd threads is pitifully low! > +# > +# > +# Each nfsd thread consumes ~1MB of memory, long-term: > +# > +# stack, kthread overhead, svc_rqst, and the rq_pages array, plus > +# temporary working space. > +# > +# Allow 32 threads for each GB of guest memory: I agree that 8 is a small number, but that multiplier seems excessive. On my 16GB test systems, that would create 512 threads. I set the nfsd thread count to 32 on my test NFS server, and I've never seen more than 10-15 threads for any test. (OK, maybe I should run tests that drive the test server harder). Plus, no distribution uses that configuration. Maybe we should stick to what's a common deployment scenario? Do you have data that shows creating more threads improves test run time? > + > +. "${TOPDIR}/.config" > + > +echo "$(( 32 * $CONFIG_LIBVIRT_MEM_MB / 1024 ))" How would this work on cloud (non-libvirt) systems? -- Chuck Lever