From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9D1E285056 for ; Thu, 26 Feb 2026 14:47:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772117262; cv=none; b=TSHy23KBVm2J1xTUaw/ME2mTmEDWwCLQZ2vQIl6E5CRG25/kpcjRyniyF0L7mkAERNhbjJAH54Y8+biTuROCOq9HhKY2MyGcOVjbI9LsQ+sC9C3iI1/pJ2xtYNTY1jfJnS0ytpTAlSInFRbsD5DkzDJFR/JKgmyqNdB4137DtQ0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772117262; c=relaxed/simple; bh=A5VWLX59xpGPw/zUBlmH3Q6b+BexIJyxH4xEkjEqkVA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=FF9cDGE1rmFdQM9gg/NTs4K8SZnQJdJb5uZ+XcexjMNkB0b55ZR3EFG3Q3x9t+bg3GIc2PMFu1v6yb660kmEUAJi9dZBfHhBkq3xYK6dsX4Ta8znXGfaFBT/7p7fOgdj0PpfmuG4DcfvuAwDuGA87eSBGoH5GgxW0EdQRVC4G+8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DYQGLYMh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DYQGLYMh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13415C116C6; Thu, 26 Feb 2026 14:47:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772117262; bh=A5VWLX59xpGPw/zUBlmH3Q6b+BexIJyxH4xEkjEqkVA=; h=From:To:Cc:Subject:Date:From; b=DYQGLYMhpl4yyDdm5xS6CtuOq9hsKQBqcWHR4ue6boqrZwf7wBEOSMCVurTleRsbv BtPNtM6oTHoRYHM9hhz8nKgPtXyuqUZc3efiqXS2L59vAhVI3DVImJVREntcfAuHAj WKh+69YmBP8NsbJRl1ZQ298Qe0UGzkAt7Bplqhf2yrAa+kxn8csPKj41NjPCWW+lKj N2haNM3wB7GmX9oeQv5TzDBYxA/RxuqY2gBA8aL8gSIBr1aBObZtP4rrUNJ3rRzVrW 5gTLw0cmXVXSpqIUHGvSDixPDW+YR7xVM2Wh4ZiayFNnJZzTc++nHCl0Iyj1ueBI37 q/vNvX97IhPiA== From: Chuck Lever To: NeilBrown , Jeff Layton , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: , Chuck Lever Subject: [PATCH v2 0/6] Optimize NFSD buffer page management Date: Thu, 26 Feb 2026 09:47:33 -0500 Message-ID: <20260226144739.193129-1-cel@kernel.org> X-Mailer: git-send-email 2.53.0 Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Chuck Lever This series solves two problems. First: NFSv3 operations have complementary Request and Response sizes. When a Request message is large, the corresponding Response message is small, and vice versa. The sum of the two message sizes is never more than the maximum transport payload size. So NFSD could get away with maintaining a single array of pages, split between the RPC send and Receive buffer. NFSv4 is not as cut and dried. An NFSv4 client may construct an NFSv4 COMPOUND that is arbitrarily complex, mixing operations that can have large Request size with operations that have a large Response size. The resulting server-side buffer size requirement can be larger than the maximum transport payload size. Therefore we must increase the allocated RPC Call landing zone and the RPC Reply construction zone to ensure that arbitrary NFSv4 COMPOUNDs can be handled. Second: Due to the above, and because NFSD can now handle payload sizes considerably larger than 1MB, the number of array entries that alloc_bulk_pages() walks through to reset the rqst page arrays after each RPC completes has increased dramatically. But we observe that the mean size of NFS requests remains smaller than a few pages. If only a few pages are consumed while processing each RPC, then traversing all of the pages in the page arrays for refills is wasted effort. The CPU cost of walking these arrays is noticeable in "perf" captures. It would be more efficient to keep track of which entries need to be refilled, since that is likely to be a small number in the most common case, and use alloc_bulk_pages() to fill only those entries. --- Changes since RFC: - Clarify a number of comments based on review (NeilBrown) - Possible NFSv3 waste is still open for discussion Chuck Lever (6): SUNRPC: Tighten bounds checking in svc_rqst_replace_page SUNRPC: Allocate a separate Reply page array SUNRPC: Handle NULL entries in svc_rqst_release_pages svcrdma: preserve rq_next_page in svc_rdma_save_io_pages SUNRPC: Track consumed rq_pages entries SUNRPC: Optimize rq_respages allocation in svc_alloc_arg include/linux/sunrpc/svc.h | 61 +++++++++++++++---------- net/sunrpc/svc.c | 59 +++++++++++++++++------- net/sunrpc/svc_xprt.c | 47 +++++++++++++++---- net/sunrpc/svcsock.c | 7 +-- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 15 ++---- net/sunrpc/xprtrdma/svc_rdma_rw.c | 1 + net/sunrpc/xprtrdma/svc_rdma_sendto.c | 6 +-- 7 files changed, 125 insertions(+), 71 deletions(-) -- 2.53.0