From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E45D1C83F1A for ; Mon, 14 Jul 2025 16:47:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81BCB6B009B; Mon, 14 Jul 2025 12:47:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C74B6B00A2; Mon, 14 Jul 2025 12:47:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68ECF6B00A7; Mon, 14 Jul 2025 12:47:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5097B6B009B for ; Mon, 14 Jul 2025 12:47:42 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BB58D8026A for ; Mon, 14 Jul 2025 16:47:41 +0000 (UTC) X-FDA: 83663451522.24.8F66BB9 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf27.hostedemail.com (Postfix) with ESMTP id 23F4D4000E for ; Mon, 14 Jul 2025 16:47:40 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="HG/f4Gdw"; spf=pass (imf27.hostedemail.com: domain of snitzer@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752511660; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ta/2PYr2QH+2onK1LFnrBHFPFSNv7cldvGxdn0mT/J0=; b=c0BRiuFxq5We1GXZo/tCmWumLt12IHZmcJcDJ/kJEmFNrd8xuin8UCTEQbpxL6FqQPL/sy E0Qu/3vMS8AUh/hsvwnLAyxc7tHzLG5JsfbgSPAHGG1iM2qLHA4AMTiIvX+zxPCIaREBd7 iDG710s5fqFruDLB9QvXqKI4g1rKZx8= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="HG/f4Gdw"; spf=pass (imf27.hostedemail.com: domain of snitzer@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752511660; a=rsa-sha256; cv=none; b=w9ADq2HpZ/p2934tJAtK36f/8mIpSi8L3AHSRkjpEYlQRgOfCc8qePXUX909sD83VDfhmE 2DvQBrkDmthmMgN26H3t3d55rFzl3EU3Tt0b0pcV5HFVyu1CqViORRAEcgjo4IJmQ7OBQs 10hyUJVNHQ65w61Ub4Tzh7xONhmICVE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 6C5D3A4E38A; Mon, 14 Jul 2025 16:47:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D91E5C4CEED; Mon, 14 Jul 2025 16:47:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752511659; bh=JIEp9aOWFSLK81pppMaZknBjhEbuP88AzmAidgrsO6Q=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=HG/f4Gdw7CdLtlzIVp4HyzB32cXcK34VPSwFvxd2y0B7Su6R2D1PJI+GEwQ0F1qAQ IntVMiYmOktivH9hlz3DuJqxz9T55CCWoN9VFwi+wjturn7C0StuqIJIhqD1kmzfQd ymOJGhoxFpe0zweHV8s37msoc+fK3SQlV+Fq3RzSrc3du4Ao0FFTawSpd397wd1nIL b5nK8sZMxfmHmwt+4us4wiJm3Zk5tPjkN+ZVX6jM28alsnwU6MEzJKgwT9sdenxodC 0x7P+IIXb/HP4GaeSe+Qur0NBWvVgluIw8P+2HWz0jbFyiYq9bujOYJLbo+bt1TjrV Q6e1xs5qZZk7Q== Date: Mon, 14 Jul 2025 12:47:37 -0400 From: Mike Snitzer To: Chuck Lever Cc: Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, linus-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH v2 6/8] NFSD: add io_cache_read controls to debugfs interface Message-ID: References: <20250708160619.64800-1-snitzer@kernel.org> <20250708160619.64800-7-snitzer@kernel.org> <6a05d14f-dbf6-4786-ba08-c57f8f4c64e6@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6a05d14f-dbf6-4786-ba08-c57f8f4c64e6@oracle.com> X-Stat-Signature: qqdo4y8rkk5byaurxe5y7chcg4zydnht X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 23F4D4000E X-Rspam-User: X-HE-Tag: 1752511660-709829 X-HE-Meta: U2FsdGVkX180/aalEtcqjvCsiVBYaolrls3jhjvrsI6sEKCHajiaOwiUqwk+hsFAZqS0TGqbc3xXz26p/+4wuYY+QjhVTCGtHaQUUvEAkejB3nq+pFbwfj1K3Nzr3M6RYox5dkcgtjuq6Vu5CepiQB4mL20+jDevobH+U9ANpblzqIEx+parZMhZNv0eHQhHR0Bkg9yAN2/wxfiWKY2SMXb8YBNCo295MOdHQWPrAdeVFu4x83V1gF6mp2HXzaXJy+KPkZ808Kx0X8q85tsRh0n2ivu9ffVLQVbVCusMFb0rYmp3wuiYhqGC1Y91KnnTL6tpK4ztMILmdKpZkbJocpWaDCdCLMU1KVh5CLn2NZaDT4iQmexWETpDrpbgq5aUxxLm0+fUvRpLaT5rfS0UXpX3R+wRSaA3rwl61pLQbOIgaz/aPl4+SBExzBRgOmtnJT4CjKEL5+7HFF6uzTox70LHvmI4RlZKjZRNdEe02r2kCM8mkQ5sebnDY7tqaLZTZN14hgSsiorpxrl9R6GlmcYVBriSMZEDPoaA9kvgNryrJX2RsywXRWnN+Hsehm5Eaw2JLnNxkZJY6l1f/CjkFENOjcSA/rq2xU01FDlF6sd9LEFnVo5FQ5i4at2zNQgdTUErA7bfT6KwEZ7uWdbLSjNawSGo/ks5fQieLx3LHvl8Fy/QZejnFoWpkZgdXLl4o725Iipevw02a3fgVJlK52WfTtr8OXOXqId2irG3CTAJg3Hcqt+R17TdYCtVK87fUKyivrSz5PfxI9L2Hc6ovWgQAjWkI8YIVLk4g3coeubnOzrIbdsPjl4wiMrVxyInV9ZzXgPVtgftlp/l97GQhVKf2pqPCHo/xfSW5LF58r4RxT2B4Ie9B/J4NH/w015ubiBs1Rb5bSOaRDHG3JgseYuTHe7IglJx5QrFg6SyHIHs4w1idnHz2OdX0CWipBHQi+JZ4g+r34SwUi2GOaf o50PaDLW gJVIsBOkoJJcjIp2Enl3XN9IUcuI/7gAom0MCrNhP2YEuO1ijon7u5nHy97HWvs1BlzG3S0xWVDCtkhf/zv2OaINEtWYb+W5HWVuk6brzmeL/BuNPe0s43MOhbpLXyP4LNzkleBPZCwdRQni9EUFJOVV+98WfxmyxdoS/CN2Zrq94X4OuTjUqZMMuRh0dye0FWV3WRmCkFaTkXx3QMGmj+pyvwPTbx9edmCAlww0jiLLhUVt7U2FJt3UEB9sZ/1GwERxY4BSF5Ufubm7mRNFEDJba8ViZDqZuIqWa8FSwQyumyPkAYzaVGTemznxkfHzg607u X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jul 10, 2025 at 06:46:37PM -0400, Chuck Lever wrote: > On 7/10/25 10:06 AM, Jeff Layton wrote: > > On Tue, 2025-07-08 at 12:06 -0400, Mike Snitzer wrote: > >> Add 'io_cache_read' to NFSD's debugfs interface so that: Any data > >> read by NFSD will either be: > >> - cached using page cache (NFSD_IO_BUFFERED=0) > >> - cached but removed from the page cache upon completion > >> (NFSD_IO_DONTCACHE=1). > >> - not cached (NFSD_IO_DIRECT=2) > >> > >> io_cache_read is 0 by default. It may be set by writing to: > >> /sys/kernel/debug/nfsd/io_cache_read > >> > >> If NFSD_IO_DONTCACHE is specified using 1, FOP_DONTCACHE must be > >> advertised as supported by the underlying filesystem (e.g. XFS), > >> otherwise all IO flagged with RWF_DONTCACHE will fail with > >> -EOPNOTSUPP. > >> > >> If NFSD_IO_DIRECT is specified using 2, the IO must be aligned > >> relative to the underlying block device's logical_block_size. Also the > >> memory buffer used to store the read must be aligned relative to the > >> underlying block device's dma_alignment. > >> > >> Signed-off-by: Mike Snitzer > >> --- > >> fs/nfsd/debugfs.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++ > >> fs/nfsd/nfsd.h | 8 +++++++ > >> fs/nfsd/vfs.c | 15 ++++++++++++++ > >> 3 files changed, 76 insertions(+) > >> > >> diff --git a/fs/nfsd/debugfs.c b/fs/nfsd/debugfs.c > >> index 84b0c8b559dc..709646af797a 100644 > >> --- a/fs/nfsd/debugfs.c > >> +++ b/fs/nfsd/debugfs.c > >> @@ -27,11 +27,61 @@ static int nfsd_dsr_get(void *data, u64 *val) > >> static int nfsd_dsr_set(void *data, u64 val) > >> { > >> nfsd_disable_splice_read = (val > 0) ? true : false; > >> + if (!nfsd_disable_splice_read) { > >> + /* > >> + * Cannot use NFSD_IO_DONTCACHE or NFSD_IO_DIRECT > >> + * if splice_read is enabled. > >> + */ > >> + nfsd_io_cache_read = NFSD_IO_BUFFERED; > >> + } > >> return 0; > >> } > >> > >> DEFINE_DEBUGFS_ATTRIBUTE(nfsd_dsr_fops, nfsd_dsr_get, nfsd_dsr_set, "%llu\n"); > >> > >> +/* > >> + * /sys/kernel/debug/nfsd/io_cache_read > >> + * > >> + * Contents: > >> + * %0: NFS READ will use buffered IO (default) > >> + * %1: NFS READ will use dontcache (buffered IO w/ dropbehind) > >> + * %2: NFS READ will use direct IO > >> + * > >> + * The default value of this setting is zero (buffered IO is > >> + * used). This setting takes immediate effect for all NFS > >> + * versions, all exports, and in all NFSD net namespaces. > >> + */ > >> + > > > > Could we switch this to use a string instead? Maybe > > buffered/dontcache/direct ? > > That thought occurred to me too, since it would make the API a little > more self-documenting, and might be a harbinger of what a future > export option might look like. > > > >> +static int nfsd_io_cache_read_get(void *data, u64 *val) > >> +{ > >> + *val = nfsd_io_cache_read; > >> + return 0; > >> +} > >> + > >> +static int nfsd_io_cache_read_set(void *data, u64 val) > >> +{ > >> + switch (val) { > >> + case NFSD_IO_DONTCACHE: > >> + case NFSD_IO_DIRECT: > >> + /* > >> + * Must disable splice_read when enabling > >> + * NFSD_IO_DONTCACHE or NFSD_IO_DIRECT. > >> + */ > >> + nfsd_disable_splice_read = true; > >> + nfsd_io_cache_read = val; > >> + break; > >> + case NFSD_IO_BUFFERED: > >> + default: > >> + nfsd_io_cache_read = NFSD_IO_BUFFERED; > >> + break; > > > > I think the default case should leave nfsd_io_cache_read alone and > > return an error. If we add new values later, and someone tries to use > > them on an old kernel, it's better to make that attempt error out. > > > > Ditto for the write side controls. > > +1 on both accounts. I started to implement this just now (so that I can kick v3 of this patchset out of the nest today) but soon found that debugfs doesn't provide string-based interface controls. See simple_attr_open() (which is used by DEFINE_DEBUGFS_ATTRIBUTE). It only allows u64 to be set/get. I'll fix the default case to return an error for now though. Once we graduate from debugfs to a proper per-export control we can impose string controls/mapping, e.g.: +static u64 nfsd_io_cache_string_to_mode(const char *nfsd_io_cache_string) +{ + u64 val = NFSD_IO_UNKNOWN; + + if (!strncmp(nfsd_io_cache_string, NFSD_IO_BUFFERED_string, + strlen(NFSD_IO_BUFFERED_string))) + val = NFSD_IO_BUFFERED; + else if (!strncmp(nfsd_io_cache_string, NFSD_IO_DONTCACHE_string, + strlen(NFSD_IO_DONTCACHE_string))) + val = NFSD_IO_DONTCACHE; + else if (!strncmp(nfsd_io_cache_string, NFSD_IO_DIRECT_string, + strlen(NFSD_IO_DIRECT_string))) + val = NFSD_IO_DIRECT; + + return val; +} + +static const char * +nfsd_io_cache_mode_to_string(const char *nfsd_io_cache_string) +{ + char *nfsd_io_cache_string; + + switch (val) { + case NFSD_IO_BUFFERED: + nfsd_io_cache_string = NFSD_IO_BUFFERED_string; + break; + case NFSD_IO_DONTCACHE: + nfsd_io_cache_string = NFSD_IO_DONTCACHE_string; + break; + case NFSD_IO_DIRECT: + nfsd_io_cache_string = NFSD_IO_DIRECT_string; + break; + case NFSD_IO_UNKNOWN: + nfsd_io_cache_string = NFSD_IO_UNKNOWN_string; + break; + } + + return nfsd_io_cache_string; +}