NFS Patch for FSCache

NFS Patch for FSCache

[Steve Dickson]

[-- Attachment #1: Type: text/plain, Size: 2089 bytes --]

Hello,

Attached is a patch that enables NFS to use David Howells'
File System Caching implementation (FSCache). Also attached
are two supplemental patches that are need to fixed two oops that
were found during debugging (Note: these patches are also
in people.redhat.com/steved/cachefs/2.6.12-rc3-mm3/)

2.6.12-rc3-mm3-nfs-fscache.patch - David and I have
been working on this for sometime now and the code
seems to be pretty solid. One issue is Trond's dislike
of how the NFS code is dependent on FSCache calls. I did
looking to changing this, but its not clear (at least to
me) how we could make things better... But that's
something that will need to be addressed.

The second issue is what we've been calling "NFS aliasing".
The fact that two mounted NFS super blocks can point to
the same page causes major fits for FSC. David has
proposed some patches to resolve this issue that are still
under review. But at this point, to stop a BUG() popping,
when a second NFS filesystem is mounted, the
2.6.12-rc3-mm3-fscache-cookie-exist.patch is needed.

The final patch 2.6.12-rc3-mm3-cachefs-wb.patch is need
to stop another BUG() from popping during NFS reads.

NFS uses FSC on a per-mount bases which means a new
mount flag 'fsc' is need to activate the caching.
Example:
     mount -t nfs4 -o fsc server:/home /mnt/server/home

(Note: people.redhat.com/steved/cachefs/util-linux/ has the
util-linux binary and source rpms with the fsc support).

To set up a mounted cachefs partition, first initialize
the disk partition by:
    echo "cachefs___" >/dev/hdg9
then mount the partition:
     mount -t cachefs /dev/hdg9 /cache-hdg9

See Documentation/filesystems/caching in the kernel
source for more details.

I'm hopeful that you'll added these patches to your tree
so they will get some much needed testing. I'm also going
to be pushing to get the caching code into a Fedora Core
kernel, but due to the  dependency on David's new vm_ops,
page_mkwrite, this might take some time...
(Note: people.redhat.com/steved/cachefs/mmpatches has all of the
current mm patches)

Comments?

steved.


[-- Attachment #2: 2.6.12-rc3-mm3-nfs-fscache.patch --]
[-- Type: text/x-patch, Size: 26573 bytes --]

This patch enables NFS to use file system caching (i.e. FSCache).
To turn this feature on you must specifiy the -o fsc mount flag
as well as have a cachefs partition mounted. 

Signed-off-by: Steve Dickson <steved@redhat.com>

--- 2.6.12-rc2-mm3/fs/nfs/file.c.orig	2005-04-23 10:13:24.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/nfs/file.c	2005-04-23 11:25:47.000000000 -0400
@@ -27,9 +27,11 @@
 #include <linux/slab.h>
 #include <linux/pagemap.h>
 #include <linux/smp_lock.h>
+#include <linux/buffer_head.h>
 
 #include <asm/uaccess.h>
 #include <asm/system.h>
+#include "nfs-fscache.h"
 
 #include "delegation.h"
 
@@ -194,6 +196,12 @@ nfs_file_sendfile(struct file *filp, lof
 	return res;
 }
 
+static int nfs_file_page_mkwrite(struct vm_area_struct *vma, struct page *page)
+{
+	wait_on_page_fs_misc(page);
+	return 0;
+}
+
 static int
 nfs_file_mmap(struct file * file, struct vm_area_struct * vma)
 {
@@ -207,6 +215,10 @@ nfs_file_mmap(struct file * file, struct
 	status = nfs_revalidate_inode(NFS_SERVER(inode), inode);
 	if (!status)
 		status = generic_file_mmap(file, vma);
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		vma->vm_ops->page_mkwrite = nfs_file_page_mkwrite;
+
 	return status;
 }
 
@@ -258,6 +270,11 @@ static int nfs_commit_write(struct file 
 	return status;
 }
 
+/*
+ * since we use page->private for our own nefarious purposes when using fscache, we have to
+ * override extra address space ops to prevent fs/buffer.c from getting confused, even though we
+ * may not have asked its opinion
+ */
 struct address_space_operations nfs_file_aops = {
 	.readpage = nfs_readpage,
 	.readpages = nfs_readpages,
@@ -269,6 +286,11 @@ struct address_space_operations nfs_file
 #ifdef CONFIG_NFS_DIRECTIO
 	.direct_IO = nfs_direct_IO,
 #endif
+#ifdef CONFIG_NFS_FSCACHE
+	.sync_page	= block_sync_page,
+	.releasepage	= nfs_releasepage,
+	.invalidatepage	= nfs_invalidatepage,
+#endif
 };
 
 /* 
--- 2.6.12-rc2-mm3/fs/nfs/inode.c.orig	2005-04-23 10:13:24.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/nfs/inode.c	2005-04-23 17:51:57.000000000 -0400
@@ -42,6 +42,8 @@
 #include "nfs4_fs.h"
 #include "delegation.h"
 
+#include "nfs-fscache.h"
+
 #define NFSDBG_FACILITY		NFSDBG_VFS
 #define NFS_PARANOIA 1
 
@@ -169,6 +171,10 @@ nfs_clear_inode(struct inode *inode)
 	cred = nfsi->cache_access.cred;
 	if (cred)
 		put_rpccred(cred);
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_clear_fscookie(nfsi);
+
 	BUG_ON(atomic_read(&nfsi->data_updates) != 0);
 }
 
@@ -503,6 +509,9 @@ nfs_fill_super(struct super_block *sb, s
 			server->namelen = NFS2_MAXNAMLEN;
 	}
 
+	if (server->flags & NFS_MOUNT_FSCACHE)
+		nfs_fill_fscookie(sb);
+
 	sb->s_op = &nfs_sops;
 	return nfs_sb_init(sb, authflavor);
 }
@@ -579,6 +588,7 @@ static int nfs_show_options(struct seq_f
 		{ NFS_MOUNT_NOAC, ",noac", "" },
 		{ NFS_MOUNT_NONLM, ",nolock", ",lock" },
 		{ NFS_MOUNT_NOACL, ",noacl", "" },
+		{ NFS_MOUNT_FSCACHE, ",fscache", "" },
 		{ 0, NULL, NULL }
 	};
 	struct proc_nfs_info *nfs_infop;
@@ -623,6 +633,9 @@ nfs_zap_caches(struct inode *inode)
 		nfsi->flags |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_DATA|NFS_INO_INVALID_ACCESS|NFS_INO_INVALID_ACL;
 	else
 		nfsi->flags |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_ACCESS|NFS_INO_INVALID_ACL;
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_zap_fscookie(nfsi);
 }
 
 static void nfs_zap_acl_cache(struct inode *inode)
@@ -770,6 +783,9 @@ nfs_fhget(struct super_block *sb, struct
 		memset(nfsi->cookieverf, 0, sizeof(nfsi->cookieverf));
 		nfsi->cache_access.cred = NULL;
 
+		if (NFS_SB(sb)->flags & NFS_MOUNT_FSCACHE)
+			nfs_fhget_fscookie(sb, nfsi);
+
 		unlock_new_inode(inode);
 	} else
 		nfs_refresh_inode(inode, fattr);
@@ -1076,6 +1092,9 @@ __nfs_revalidate_inode(struct nfs_server
 				(long long)NFS_FILEID(inode));
 		/* This ensures we revalidate dentries */
 		nfsi->cache_change_attribute++;
+
+		if (server->flags & NFS_MOUNT_FSCACHE)
+			nfs_renew_fscookie(server, nfsi);
 	}
 	if (flags & NFS_INO_INVALID_ACL)
 		nfs_zap_acl_cache(inode);
@@ -1515,6 +1534,14 @@ static struct super_block *nfs_get_sb(st
 		goto out_err;
 	}
 
+#ifndef CONFIG_NFS_FSCACHE
+	if (data->flags & NFS_MOUNT_FSCACHE) {
+		printk(KERN_WARNING "NFS: kernel not compiled with CONFIG_NFS_FSCACHE\n");
+		kfree(server);
+		return ERR_PTR(-EINVAL);
+	}
+#endif
+
 	s = sget(fs_type, nfs_compare_super, nfs_set_super, server);
 	if (IS_ERR(s) || s->s_root)
 		goto out_rpciod_down;
@@ -1542,6 +1569,9 @@ static void nfs_kill_super(struct super_
 
 	kill_anon_super(s);
 
+	if (server->flags & NFS_MOUNT_FSCACHE)
+		nfs_kill_fscookie(server);
+
 	if (server->client != NULL && !IS_ERR(server->client))
 		rpc_shutdown_client(server->client);
 	if (server->client_sys != NULL && !IS_ERR(server->client_sys))
@@ -1760,6 +1790,9 @@ static int nfs4_fill_super(struct super_
 
 	sb->s_time_gran = 1;
 
+	if (server->flags & NFS4_MOUNT_FSCACHE)
+		nfs4_fill_fscookie(sb);
+
 	sb->s_op = &nfs4_sops;
 	err = nfs_sb_init(sb, authflavour);
 	if (err == 0)
@@ -1903,6 +1936,9 @@ static void nfs4_kill_super(struct super
 	nfs_return_all_delegations(sb);
 	kill_anon_super(sb);
 
+	if (server->flags & NFS_MOUNT_FSCACHE)
+		nfs_kill_fscookie(server);
+
 	nfs4_renewd_prepare_shutdown(server);
 
 	if (server->client != NULL && !IS_ERR(server->client))
@@ -2021,6 +2057,11 @@ static int __init init_nfs_fs(void)
 {
 	int err;
 
+	/* we want to be able to cache */
+	err = nfs_register_netfs();
+	if (err < 0)
+		goto out5;
+
 	err = nfs_init_nfspagecache();
 	if (err)
 		goto out4;
@@ -2068,6 +2109,9 @@ out2:
 out3:
 	nfs_destroy_nfspagecache();
 out4:
+	nfs_unregister_netfs();
+out5:
+
 	return err;
 }
 
@@ -2080,6 +2124,7 @@ static void __exit exit_nfs_fs(void)
 	nfs_destroy_readpagecache();
 	nfs_destroy_inodecache();
 	nfs_destroy_nfspagecache();
+	nfs_unregister_netfs();
 #ifdef CONFIG_PROC_FS
 	rpc_proc_unregister("nfs");
 #endif
--- 2.6.12-rc2-mm3/fs/nfs/Makefile.orig	2005-04-23 10:13:24.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/nfs/Makefile	2005-04-23 11:25:47.000000000 -0400
@@ -13,4 +13,5 @@ nfs-$(CONFIG_NFS_V4)	+= nfs4proc.o nfs4x
 			   delegation.o idmap.o \
 			   callback.o callback_xdr.o callback_proc.o
 nfs-$(CONFIG_NFS_DIRECTIO) += direct.o
+nfs-$(CONFIG_NFS_FSCACHE) += nfs-fscache.o
 nfs-objs		:= $(nfs-y)
--- /dev/null	2005-03-28 14:47:10.233040208 -0500
+++ 2.6.12-rc2-mm3/fs/nfs/nfs-fscache.c	2005-04-23 15:14:02.000000000 -0400
@@ -0,0 +1,191 @@
+/* nfs-fscache.c: NFS filesystem cache interface
+ *
+ * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+
+#include <linux/config.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/nfs_fs.h>
+#include <linux/nfs_fs_sb.h>
+
+#include "nfs-fscache.h"
+
+#define NFS_CACHE_FH_INDEX_SIZE sizeof(struct nfs_fh)
+
+/*
+ * the root index is
+ */
+static struct fscache_page *nfs_cache_get_page_token(struct page *page);
+
+static struct fscache_netfs_operations nfs_cache_ops = {
+	.get_page_token	= nfs_cache_get_page_token,
+};
+
+struct fscache_netfs nfs_cache_netfs = {
+	.name			= "nfs",
+	.version		= 0,
+	.ops			= &nfs_cache_ops,
+};
+
+/*
+ * the root index for the filesystem is defined by nfsd IP address and ports
+ */
+static fscache_match_val_t nfs_cache_server_match(void *target,
+						  const void *entry);
+static void nfs_cache_server_update(void *source, void *entry);
+
+struct fscache_index_def nfs_cache_server_index_def = {
+	.name			= "servers",
+	.data_size		= 18,
+	.keys[0]		= { FSCACHE_INDEX_KEYS_IPV6ADDR, 16 },
+	.keys[1]		= { FSCACHE_INDEX_KEYS_BIN, 2 },
+	.match			= nfs_cache_server_match,
+	.update			= nfs_cache_server_update,
+};
+
+/*
+ * the primary index for each server is simply made up of a series of NFS file
+ * handles
+ */
+static fscache_match_val_t nfs_cache_fh_match(void *target, const void *entry);
+static void nfs_cache_fh_update(void *source, void *entry);
+
+struct fscache_index_def nfs_cache_fh_index_def = {
+	.name			= "fh",
+	.data_size		= NFS_CACHE_FH_INDEX_SIZE,
+	.keys[0]		= { FSCACHE_INDEX_KEYS_BIN_SZ2,
+				    sizeof(struct nfs_fh) },
+	.match			= nfs_cache_fh_match,
+	.update			= nfs_cache_fh_update,
+};
+
+/*
+ * get a page token for the specified page
+ * - the token will be attached to page->private and PG_private will be set on
+ *   the page
+ */
+static struct fscache_page *nfs_cache_get_page_token(struct page *page)
+{
+	return fscache_page_get_private(page, GFP_NOIO);
+}
+
+static const uint8_t nfs_cache_ipv6_wrapper_for_ipv4[12] = {
+	[0 ... 9]	= 0x00,
+	[10 ... 11]	= 0xff
+};
+
+/*
+ * match a server record obtained from the cache
+ */
+static fscache_match_val_t nfs_cache_server_match(void *target,
+						  const void *entry)
+{
+	struct nfs_server *server = target;
+	const uint8_t *data = entry;
+
+	switch (server->addr.sin_family) {
+	case AF_INET:
+		if (memcmp(data + 0,
+			   &nfs_cache_ipv6_wrapper_for_ipv4,
+			   12) != 0)
+			break;
+
+		if (memcmp(data + 12, &server->addr.sin_addr, 4) != 0)
+			break;
+
+		if (memcmp(data + 16, &server->addr.sin_port, 2) != 0)
+			break;
+
+		return FSCACHE_MATCH_SUCCESS;
+
+	case AF_INET6:
+		if (memcmp(data + 0, &server->addr.sin_addr, 16) != 0)
+			break;
+
+		if (memcmp(data + 16, &server->addr.sin_port, 2) != 0)
+			break;
+
+		return FSCACHE_MATCH_SUCCESS;
+
+	default:
+		break;
+	}
+
+	return FSCACHE_MATCH_FAILED;
+}
+
+/*
+ * update a server record in the cache
+ */
+static void nfs_cache_server_update(void *source, void *entry)
+{
+	struct nfs_server *server = source;
+	uint8_t *data = entry;
+
+	switch (server->addr.sin_family) {
+	case AF_INET:
+		memcpy(data + 0, &nfs_cache_ipv6_wrapper_for_ipv4, 12);
+		memcpy(data + 12, &server->addr.sin_addr, 4);
+		memcpy(data + 16, &server->addr.sin_port, 2);
+		return;
+
+	case AF_INET6:
+		memcpy(data + 0, &server->addr.sin_addr, 16);
+		memcpy(data + 16, &server->addr.sin_port, 2);
+		return;
+
+	default:
+		return;
+	}
+}
+
+/*
+ * match a file handle record obtained from the cache
+ */
+static fscache_match_val_t nfs_cache_fh_match(void *target, const void *entry)
+{
+	struct nfs_inode *nfsi = target;
+	const uint8_t *data = entry;
+	uint16_t nsize;
+
+	/* check the file handle matches */
+	memcpy(&nsize, data, 2);
+	nsize = ntohs(nsize);
+
+	if (nsize <= NFS_CACHE_FH_INDEX_SIZE && nfsi->fh.size == nsize) {
+		if (memcmp(data + 2, nfsi->fh.data, nsize) == 0) {
+			return FSCACHE_MATCH_SUCCESS;
+		}
+	}
+
+	return FSCACHE_MATCH_FAILED;
+}
+
+/*
+ * update a fh record in the cache
+ */
+static void nfs_cache_fh_update(void *source, void *entry)
+{
+	struct nfs_inode *nfsi = source;
+	uint16_t nsize;
+	uint8_t *data = entry;
+
+	BUG_ON(nfsi->fh.size > NFS_CACHE_FH_INDEX_SIZE - 2);
+
+	/* set the file handle */
+	nsize = htons(nfsi->fh.size);
+	memcpy(data, &nsize, 2);
+	memcpy(data + 2, &nfsi->fh.data, nfsi->fh.size);
+	memset(data + 2 + nfsi->fh.size,
+	       FSCACHE_INDEX_DEADFILL_PATTERN,
+	       NFS_CACHE_FH_INDEX_SIZE - 2 - nfsi->fh.size);
+}
--- /dev/null	2005-03-28 14:47:10.233040208 -0500
+++ 2.6.12-rc2-mm3/fs/nfs/nfs-fscache.h	2005-04-23 17:51:15.000000000 -0400
@@ -0,0 +1,158 @@
+/* nfs-fscache.h: NFS filesystem cache interface definitions
+ *
+ * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _NFS_FSCACHE_H
+#define _NFS_FSCACHE_H
+
+#include <linux/nfs_mount.h>
+#include <linux/nfs4_mount.h>
+#include <linux/fscache.h>
+
+#ifdef CONFIG_NFS_FSCACHE
+#ifndef CONFIG_FSCACHE
+#error "CONFIG_NFS_FSCACHE is defined but not CONFIG_FSCACHE"
+#endif
+
+extern struct fscache_netfs nfs_cache_netfs;
+extern struct fscache_index_def nfs_cache_server_index_def;
+extern struct fscache_index_def nfs_cache_fh_index_def;
+
+extern int nfs_invalidatepage(struct page *, unsigned long);
+extern int nfs_releasepage(struct page *, int);
+extern int nfs_mkwrite(struct page *);
+
+static inline void 
+nfs_renew_fscookie(struct nfs_server *server, struct nfs_inode *nfsi)
+{
+	struct fscache_cookie *old =  nfsi->fscache;
+
+	/* retire the current fscache cache and get a new one */
+	fscache_relinquish_cookie(nfsi->fscache, 1);
+	nfsi->fscache = fscache_acquire_cookie(server->fscache, NULL, nfsi);
+
+	dfprintk(FSCACHE,
+		"NFS: revalidation new cookie (0x%p/0x%p/0x%p/0x%p)\n",
+		server, nfsi, old, nfsi->fscache);
+
+	return;
+}
+static inline void
+nfs4_fill_fscookie(struct super_block *sb)
+{
+	struct nfs_server *server = NFS_SB(sb);
+
+	/* create a cache index for looking up filehandles */
+	server->fscache = fscache_acquire_cookie(nfs_cache_netfs.primary_index,
+			       &nfs_cache_fh_index_def, server);
+	if (server->fscache == NULL) {
+		printk(KERN_WARNING "NFS4: No Fscache cookie. Turning Fscache off!\n");
+	} else /* reuse the NFS mount option */
+		server->flags |= NFS_MOUNT_FSCACHE;
+
+	dfprintk(FSCACHE,"NFS: nfs4 cookie (0x%p,0x%p/0x%p)\n", 
+		sb, server, server->fscache);
+
+	return;
+}
+static inline void
+nfs_fill_fscookie(struct super_block *sb)
+{
+	struct nfs_server *server = NFS_SB(sb);
+
+	/* create a cache index for looking up filehandles */
+	server->fscache = fscache_acquire_cookie(nfs_cache_netfs.primary_index,
+			      &nfs_cache_fh_index_def, server);
+	if (server->fscache == NULL) {
+		server->flags &= ~NFS_MOUNT_FSCACHE;
+		printk(KERN_WARNING "NFS: No Fscache cookie. Turning Fscache off!\n");
+	}
+	dfprintk(FSCACHE,"NFS: cookie (0x%p/0x%p/0x%p)\n", 
+		sb, server, server->fscache);
+
+	return;
+}
+static inline void
+nfs_fhget_fscookie(struct super_block *sb, struct nfs_inode *nfsi)
+{
+	struct nfs_server *server = NFS_SB(sb);
+
+	nfsi->fscache = fscache_acquire_cookie(server->fscache, NULL, nfsi);
+	if (server->fscache == NULL)
+		printk(KERN_WARNING "NFS: NULL FScache cookie: sb 0x%p nfsi 0x%p\n", sb, nfsi);
+
+	dfprintk(FSCACHE, "NFS: fhget new cookie (0x%p/0x%p/0x%p)\n",
+		sb, nfsi, nfsi->fscache);
+
+	return;
+}
+static inline void 
+nfs_kill_fscookie(struct nfs_server *server)
+{
+	dfprintk(FSCACHE,"NFS: killing cookie (0x%p/0x%p)\n",
+		server, server->fscache);
+
+	fscache_relinquish_cookie(server->fscache, 0);
+	server->fscache = NULL;
+
+	return;
+}
+static inline void
+nfs_clear_fscookie(struct nfs_inode *nfsi)
+{
+	dfprintk(FSCACHE, "NFS: clear cookie (0x%p/0x%p)\n",
+			nfsi, nfsi->fscache);
+
+	fscache_relinquish_cookie(nfsi->fscache, 0);
+	nfsi->fscache = NULL;
+
+	return;
+}
+static inline void
+nfs_zap_fscookie(struct nfs_inode *nfsi)
+{
+	dfprintk(FSCACHE,"NFS: zapping cookie (0x%p/0x%p)\n",
+		nfsi, nfsi->fscache);
+
+	fscache_relinquish_cookie(nfsi->fscache, 1);
+	nfsi->fscache = NULL;
+
+	return;
+}
+static inline int
+nfs_register_netfs(void)
+{
+	int err;
+
+	err = fscache_register_netfs(&nfs_cache_netfs, &nfs_cache_server_index_def);
+
+	return err;
+}
+static inline void
+nfs_unregister_netfs(void)
+{
+	fscache_unregister_netfs(&nfs_cache_netfs);
+
+	return;
+}
+#else
+static inline void nfs_fill_fscookie(struct super_block *sb) {}
+static inline void nfs_fhget_fscookie(struct super_block *sb, struct nfs_inode *nfsi) {}
+static inline void nfs4_fill_fscookie(struct super_block *sb) {}
+static inline void nfs_kill_fscookie(struct nfs_server *server) {}
+static inline void nfs_clear_fscookie(struct nfs_inode *nfsi) {}
+static inline void nfs_zap_fscookie(struct nfs_inode *nfsi) {}
+static inline void 
+	nfs_renew_fscookie(struct nfs_server *server, struct nfs_inode *nfsi) {}
+static inline int nfs_register_netfs() { return 0; }
+static inline void nfs_unregister_netfs() {}
+
+#endif
+#endif /* _NFS_FSCACHE_H */
--- 2.6.12-rc2-mm3/fs/nfs/read.c.orig	2005-04-23 10:13:25.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/nfs/read.c	2005-04-23 11:25:47.000000000 -0400
@@ -27,6 +27,7 @@
 #include <linux/sunrpc/clnt.h>
 #include <linux/nfs_fs.h>
 #include <linux/nfs_page.h>
+#include <linux/nfs_mount.h>
 #include <linux/smp_lock.h>
 
 #include <asm/system.h>
@@ -73,6 +74,47 @@ int nfs_return_empty_page(struct page *p
 	return 0;
 }
 
+#ifdef CONFIG_NFS_FSCACHE
+/*
+ * store a newly fetched page in fscache
+ */
+static void
+nfs_readpage_to_fscache_complete(void *cookie_data, struct page *page, void *data, int error)
+{
+	dprintk("NFS:     readpage_to_fscache_complete (%p/%p/%p/%d)\n",
+		cookie_data, page, data, error);
+
+	end_page_fs_misc(page);
+}
+
+static inline void
+nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)
+{
+	int ret;
+
+	dprintk("NFS: readpage_to_fscache(0x%p/0x%p/0x%p/%d)\n",
+		NFS_I(inode)->fscache, page, inode, sync);
+
+	SetPageFsMisc(page);
+	ret = fscache_write_page(NFS_I(inode)->fscache, page,
+		nfs_readpage_to_fscache_complete, NULL, GFP_KERNEL);
+	if (ret != 0) {
+		dprintk("NFS:     readpage_to_fscache: error %d\n", ret);
+		fscache_uncache_page(NFS_I(inode)->fscache, page);
+		ClearPageFsMisc(page);
+	}
+
+	unlock_page(page);
+}
+#else
+static inline void
+nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)
+{
+	BUG();
+}
+#endif
+
+
 /*
  * Read a page synchronously.
  */
@@ -149,6 +191,13 @@ static int nfs_readpage_sync(struct nfs_
 		ClearPageError(page);
 	result = 0;
 
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_readpage_to_fscache(inode, page, 1);
+	else
+		unlock_page(page);
+
+	return result;
+
 io_error:
 	unlock_page(page);
 	nfs_readdata_free(rdata);
@@ -180,7 +229,13 @@ static int nfs_readpage_async(struct nfs
 
 static void nfs_readpage_release(struct nfs_page *req)
 {
-	unlock_page(req->wb_page);
+	struct inode *d_inode = req->wb_context->dentry->d_inode;
+
+	if ((NFS_SERVER(d_inode)->flags & NFS_MOUNT_FSCACHE) && 
+			PageUptodate(req->wb_page))
+		nfs_readpage_to_fscache(d_inode, req->wb_page, 0);
+	else
+		unlock_page(req->wb_page);
 
 	nfs_clear_request(req);
 	nfs_release_request(req);
@@ -477,6 +532,67 @@ void nfs_readpage_result(struct rpc_task
 	data->complete(data, status);
 }
 
+
+/*
+ * Read a page through the on-disc cache if possible
+ */
+#ifdef CONFIG_NFS_FSCACHE
+static void
+nfs_readpage_from_fscache_complete(void *cookie_data, struct page *page, void *data, int error)
+{
+	dprintk("NFS: readpage_from_fscache_complete (0x%p/0x%p/0x%p/%d)\n",
+		cookie_data, page, data, error);
+
+	if (error)
+		SetPageError(page);
+	else
+		SetPageUptodate(page);
+
+	unlock_page(page);
+}
+
+static inline int
+nfs_readpage_from_fscache(struct inode *inode, struct page *page)
+{
+	struct fscache_page *pageio;
+	int ret;
+
+	dprintk("NFS: readpage_from_fscache(0x%p/0x%p/0x%p)\n",
+		NFS_I(inode)->fscache, page, inode);
+
+	pageio = fscache_page_get_private(page, GFP_NOIO);
+	if (IS_ERR(pageio)) {
+		 dprintk("NFS:     fscache_page_get_private error %ld\n", PTR_ERR(pageio));
+		return PTR_ERR(pageio);
+	}
+
+	ret = fscache_read_or_alloc_page(NFS_I(inode)->fscache,
+					 page,
+					 nfs_readpage_from_fscache_complete,
+					 NULL,
+					 GFP_KERNEL);
+
+	switch (ret) {
+	case 1: /* read BIO submitted and wb-journal entry found */
+		BUG();
+		
+	case 0: /* read BIO submitted (page in fscache) */
+		return ret;
+
+	case -ENOBUFS: /* inode not in cache */
+	case -ENODATA: /* page not in cache */
+		dprintk("NFS:     fscache_read_or_alloc_page error %d\n", ret);
+		return 1;
+
+	default:
+		return ret;
+	}
+}
+#else
+static inline int 
+nfs_readpage_from_fscache(struct inode *inode, struct page *page) { return 1; }
+#endif
+
 /*
  * Read a page over NFS.
  * We read the page synchronously in the following case:
@@ -510,6 +626,13 @@ int nfs_readpage(struct file *file, stru
 		ctx = get_nfs_open_context((struct nfs_open_context *)
 				file->private_data);
 	if (!IS_SYNC(inode)) {
+		if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE) {
+			error = nfs_readpage_from_fscache(inode, page);
+			if (error < 0)
+				goto out_error;
+			if (error == 0)
+				return error;
+		}
 		error = nfs_readpage_async(ctx, inode, page);
 		goto out;
 	}
@@ -540,6 +663,15 @@ readpage_async_filler(void *data, struct
 	unsigned int len;
 
 	nfs_wb_page(inode, page);
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE) {
+		int error = nfs_readpage_from_fscache(inode, page);
+		if (error < 0)
+			return error;
+		if (error == 0)
+			return error;
+	}
+
 	len = nfs_page_length(inode, page);
 	if (len == 0)
 		return nfs_return_empty_page(page);
@@ -613,3 +745,61 @@ void nfs_destroy_readpagecache(void)
 	if (kmem_cache_destroy(nfs_rdata_cachep))
 		printk(KERN_INFO "nfs_read_data: not all structures were freed\n");
 }
+
+#ifdef CONFIG_NFS_FSCACHE
+int nfs_invalidatepage(struct page *page, unsigned long offset)
+{
+	int ret = 1;
+	struct nfs_server *server = NFS_SERVER(page->mapping->host);
+
+	BUG_ON(!PageLocked(page));
+
+	if (server->flags & NFS_MOUNT_FSCACHE) {
+		if (PagePrivate(page)) {
+			struct nfs_inode *nfsi = NFS_I(page->mapping->host);
+
+			dfprintk(PAGECACHE,"NFS: fscache invalidatepage (0x%p/0x%p/0x%p)\n",
+				nfsi->fscache, page, nfsi);
+
+			fscache_uncache_page(nfsi->fscache, page);
+
+			if (offset == 0) {
+				BUG_ON(!PageLocked(page));
+				ret = 0;
+				if (!PageWriteback(page))
+					ret = page->mapping->a_ops->releasepage(page, 0);
+			}
+		}
+	} else
+		ret = 0;
+
+	return ret;
+}
+int nfs_releasepage(struct page *page, int gfp_flags)
+{
+	struct fscache_page *pageio;
+	struct nfs_server *server = NFS_SERVER(page->mapping->host);
+
+	if (server->flags & NFS_MOUNT_FSCACHE && PagePrivate(page)) {
+		struct nfs_inode *nfsi = NFS_I(page->mapping->host);
+
+		dfprintk(PAGECACHE,"NFS: fscache releasepage (0x%p/0x%p/0x%p)\n",
+			nfsi->fscache, page, nfsi);
+
+		fscache_uncache_page(nfsi->fscache, page);
+		pageio = (struct fscache_page *) page->private;
+		page->private = 0;
+		ClearPagePrivate(page);
+
+		if (pageio)
+			kfree(pageio);
+	}
+
+	return 0;
+}
+int nfs_mkwrite(struct page *page)
+{
+	wait_on_page_fs_misc(page);
+	return 0;
+}
+#endif
--- 2.6.12-rc2-mm3/fs/nfs/write.c.orig	2005-04-23 10:13:25.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/nfs/write.c	2005-04-23 18:07:11.000000000 -0400
@@ -255,6 +255,38 @@ static int wb_priority(struct writeback_
 }
 
 /*
+ * store an updated page in fscache
+ */
+#ifdef CONFIG_NFS_FSCACHE
+static void
+nfs_writepage_to_fscache_complete(void *cookie_data, struct page *page, void *data, int error)
+{
+	/* really need to synchronise the end of writeback, probably using a page flag */
+}
+static inline void
+nfs_writepage_to_fscache(struct inode *inode, struct page *page)
+{
+	int ret; 
+
+	dprintk("NFS: writepage_to_fscache (0x%p/0x%p/0x%p)\n",
+		NFS_I(inode)->fscache, page, inode);
+
+	ret =  fscache_write_page(NFS_I(inode)->fscache, page,
+		nfs_writepage_to_fscache_complete, NULL, GFP_KERNEL);
+	if (ret != 0) {
+		dprintk("NFS:    fscache_write_page error %d\n", ret);
+		fscache_uncache_page(NFS_I(inode)->fscache, page);
+	}
+}
+#else
+static inline void
+nfs_writepage_to_fscache(struct inode *inode, struct page *page)
+{
+	BUG();
+}
+#endif
+
+/*
  * Write an mmapped page to the server.
  */
 int nfs_writepage(struct page *page, struct writeback_control *wbc)
@@ -299,6 +331,10 @@ do_it:
 		err = -EBADF;
 		goto out;
 	}
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_writepage_to_fscache(inode, page);
+
 	lock_kernel();
 	if (!IS_SYNC(inode) && inode_referenced) {
 		err = nfs_writepage_async(ctx, inode, page, 0, offset);
--- 2.6.12-rc2-mm3/fs/Kconfig.orig	2005-04-23 10:13:23.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/Kconfig	2005-04-23 11:25:48.000000000 -0400
@@ -1456,6 +1456,13 @@ config NFS_V4
 
 	  If unsure, say N.
 
+config NFS_FSCACHE
+	bool "Provide NFS client caching support (EXPERIMENTAL)"
+	depends on NFS_FS && FSCACHE && EXPERIMENTAL
+	help
+	  Say Y here if you want NFS data to be cached locally on disc through
+	  the general filesystem cache manager
+
 config NFS_DIRECTIO
 	bool "Allow direct I/O on NFS files (EXPERIMENTAL)"
 	depends on NFS_FS && EXPERIMENTAL
--- 2.6.12-rc2-mm3/include/linux/nfs_fs.h.orig	2005-04-23 10:13:28.000000000 -0400
+++ 2.6.12-rc2-mm3/include/linux/nfs_fs.h	2005-04-23 15:27:22.000000000 -0400
@@ -29,6 +29,7 @@
 #include <linux/nfs_xdr.h>
 #include <linux/rwsem.h>
 #include <linux/mempool.h>
+#include <linux/fscache.h>
 
 /*
  * Enable debugging support for nfs client.
@@ -184,6 +185,11 @@ struct nfs_inode {
 	int			 delegation_state;
 	struct rw_semaphore	rwsem;
 #endif /* CONFIG_NFS_V4*/
+
+#ifdef CONFIG_NFS_FSCACHE
+	struct fscache_cookie	*fscache;
+#endif
+
 	struct inode		vfs_inode;
 };
 
@@ -564,6 +570,7 @@ extern void * nfs_root_data(void);
 #define NFSDBG_FILE		0x0040
 #define NFSDBG_ROOT		0x0080
 #define NFSDBG_CALLBACK		0x0100
+#define NFSDBG_FSCACHE		0x0200
 #define NFSDBG_ALL		0xFFFF
 
 #ifdef __KERNEL__
--- 2.6.12-rc2-mm3/include/linux/nfs_fs_sb.h.orig	2005-04-23 10:13:28.000000000 -0400
+++ 2.6.12-rc2-mm3/include/linux/nfs_fs_sb.h	2005-04-23 11:25:48.000000000 -0400
@@ -3,6 +3,7 @@
 
 #include <linux/list.h>
 #include <linux/backing-dev.h>
+#include <linux/fscache.h>
 
 /*
  * NFS client parameters stored in the superblock.
@@ -47,6 +48,10 @@ struct nfs_server {
 						   that are supported on this
 						   filesystem */
 #endif
+
+#ifdef CONFIG_NFS_FSCACHE
+	struct fscache_cookie	*fscache;	/* cache cookie */
+#endif
 };
 
 /* Server capabilities */
--- 2.6.12-rc2-mm3/include/linux/nfs_mount.h.orig	2005-04-23 10:13:28.000000000 -0400
+++ 2.6.12-rc2-mm3/include/linux/nfs_mount.h	2005-04-23 11:25:48.000000000 -0400
@@ -61,6 +61,7 @@ struct nfs_mount_data {
 #define NFS_MOUNT_NOACL		0x0800	/* 4 */
 #define NFS_MOUNT_STRICTLOCK	0x1000	/* reserved for NFSv4 */
 #define NFS_MOUNT_SECFLAVOUR	0x2000	/* 5 */
+#define NFS_MOUNT_FSCACHE		0x3000
 #define NFS_MOUNT_FLAGMASK	0xFFFF
 
 #endif
--- 2.6.12-rc2-mm3/include/linux/nfs4_mount.h.orig	2005-03-02 02:38:09.000000000 -0500
+++ 2.6.12-rc2-mm3/include/linux/nfs4_mount.h	2005-04-23 11:25:48.000000000 -0400
@@ -65,6 +65,7 @@ struct nfs4_mount_data {
 #define NFS4_MOUNT_NOCTO	0x0010	/* 1 */
 #define NFS4_MOUNT_NOAC		0x0020	/* 1 */
 #define NFS4_MOUNT_STRICTLOCK	0x1000	/* 1 */
+#define NFS4_MOUNT_FSCACHE	0x2000	/* 1 */
 #define NFS4_MOUNT_FLAGMASK	0xFFFF
 
 #endif

[-- Attachment #3: 2.6.12-rc3-mm3-fscache-cookie-exist.patch --]
[-- Type: text/x-patch, Size: 669 bytes --]

Fails a second NFS mount with EEXIST instead of oops. 

Signed-off-by: Steve Dickson <steved@redhat.com>

--- 2.6.12-rc3-mm3/fs/fscache/cookie.c.orig	2005-05-07 09:30:28.000000000 -0400
+++ 2.6.12-rc3-mm3/fs/fscache/cookie.c	2005-05-07 11:01:39.000000000 -0400
@@ -452,7 +452,11 @@ static int fscache_search_for_object(str
 		cache->ops->lock_node(node);
 
 		/* a node should only ever be attached to one cookie */
-		BUG_ON(!list_empty(&node->cookie_link));
+		if (!list_empty(&node->cookie_link)) {
+			cache->ops->unlock_node(node);
+			ret = -EEXIST;
+			goto error;
+		}
 
 		/* attach the node to the cache's node list */
 		if (list_empty(&node->cache_link)) {

[-- Attachment #4: 2.6.12-rc2-mm3-cachefs-wb.patch --]
[-- Type: text/x-patch, Size: 452 bytes --]

--- 2.6.12-rc2-mm3/fs/cachefs/journal.c.save	2005-04-27 08:06:03.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/cachefs/journal.c	2005-05-03 11:11:17.000000000 -0400
@@ -682,6 +682,7 @@ static inline void cachefs_trans_batch_p
 		list_add_tail(&block->batch_link, plist);
 		block->writeback = block->page;
 		get_page(block->writeback);
+		SetPageWriteback(block->writeback);
 
 		/* make sure DMA can reach the data */
 		flush_dcache_page(block->writeback);

[-- Attachment #5: Type: text/plain, Size: 0 bytes --]

Re: NFS Patch for FSCache

[Andrew Morton]

Steve Dickson <SteveD@redhat.com> wrote:
>
> Attached is a patch that enables NFS to use David Howells'
> File System Caching implementation (FSCache).

Do you have any performance results for this?

Re: NFS Patch for FSCache

[Steve Dickson]

Andrew Morton wrote:
> Steve Dickson <SteveD@redhat.com> wrote:
> 
>>Attached is a patch that enables NFS to use David Howells'
>>File System Caching implementation (FSCache).
> 
> 
> Do you have any performance results for this?
I haven't done any formal performance testing, but from
the functionality testing I've done, I've seen
a ~20% increase in reads speed (verses otw reads).
Mainly due to the fact NFS only needs to do getattrs
and such when the data is cached. But buyer beware...
this a very rough number, so mileage may very. ;-)

I don't have a number for writes, (maybe David does)
but I'm sure there will be a penalty to cache that
data, but its something that can be improve over time.

But the real saving, imho, is the fact those
reads were measured after the filesystem was
umount then remounted. So system wise, there
should be some gain due to the fact that NFS
is not using the network....

steved.

Re: [Linux-cachefs] Re: NFS Patch for FSCache

[David Howells]

Steve Dickson <SteveD@redhat.com> wrote:

> But the real saving, imho, is the fact those reads were measured after the
> filesystem was umount then remounted. So system wise, there should be some
> gain due to the fact that NFS is not using the network....

I tested md5sum read speed also. My testbox is a dual 200MHz PPro. It's got
128MB of RAM. I've got a 100MB file on the NFS server for it to read.

	No Cache:	~14s
	Cold Cache:	~15s
	Warm Cache:	~2s

Now these numbers are approximate because they're from memory.

Note that a cold cache is worse than no cache because CacheFS (a) has to check
the disk before NFS goes to the server, and (b) has to journal the allocations
of new data blocks. It may also have to wait whilst pages are written to disk
before it can get new ones rather than just dropping them (100MB is big enough
wrt 128MB that this will happen) and 100MB is sufficient to cause it to start
using single- and double-indirection pointers to find its blocks on disk,
though these are cached in the page cache.

David

Re: Re: NFS Patch for FSCache

[Troy Benjegerdes]

On Tue, May 10, 2005 at 08:12:51PM +0100, David Howells wrote:
> 
> Steve Dickson <SteveD@redhat.com> wrote:
> 
> > But the real saving, imho, is the fact those reads were measured after the
> > filesystem was umount then remounted. So system wise, there should be some
> > gain due to the fact that NFS is not using the network....
> 
> I tested md5sum read speed also. My testbox is a dual 200MHz PPro. It's got
> 128MB of RAM. I've got a 100MB file on the NFS server for it to read.
> 
> 	No Cache:	~14s
> 	Cold Cache:	~15s
> 	Warm Cache:	~2s
> 
> Now these numbers are approximate because they're from memory.
> 
> Note that a cold cache is worse than no cache because CacheFS (a) has to check
> the disk before NFS goes to the server, and (b) has to journal the allocations
> of new data blocks. It may also have to wait whilst pages are written to disk
> before it can get new ones rather than just dropping them (100MB is big enough
> wrt 128MB that this will happen) and 100MB is sufficient to cause it to start
> using single- and double-indirection pointers to find its blocks on disk,
> though these are cached in the page cache.

How big was the cachefs filesystem?

Now try reading a 1GB file over nfs..

I have found (with openafs), that I either need a really small cache, or
a really big one.. The bigger the openafs cache gets, the slower it
goes. The only place i run with a > 1GB openafs cache is on an imap
server that has an 8gb cache for maildirs.

Re: Re: NFS Patch for FSCache

[David Howells]

Troy Benjegerdes <hozer@hozed.org> wrote:

> How big was the cachefs filesystem?

Several Gig. I don't remember how big, but the disk I tried it on is totally
kaput unfortunately.

> Now try reading a 1GB file over nfs..

I'll give that a go at some point. However, I suspect that any size over twice
the amount of pagecache available is going to scale fairly consistently until
you start hitting the lid on the cache. I say twice because firstly you fill
the pagecache with pages and start throwing them at the disk, and then you
have to start on a rolling process of waiting for those to hit the disk before
evicting them from the pagecache, which isn't going to get going smoothly
until you've ejected the original load of pages.

> I have found (with openafs), that I either need a really small cache, or
> a really big one.. The bigger the openafs cache gets, the slower it
> goes. The only place i run with a > 1GB openafs cache is on an imap
> server that has an 8gb cache for maildirs.

What filesystem underlies your OpenAFS cache? OpenAFS doesn't actually do its
own file to disk management within the cache, but uses a host filesystem for
that.

David

Re: NFS Patch for FSCache

[Steve Dickson]

[-- Attachment #1: Type: text/plain, Size: 349 bytes --]

I notice that a number NFS patches when into
2.6.12-rc6-mm1 so I wanted to make sure the patches I
posted, which allow NFS to use cachefs, had not
become stale. It turns out they didn't and they
still work as they did in rc3-mm3... But I figured
I would repost them anyways in hopes to get them
reviewed and accepted in the -mm tree....


steved.



[-- Attachment #2: 2.6.12-rc6-mm1-nfs-fscache.patch --]
[-- Type: text/x-patch, Size: 26574 bytes --]

This patch enables NFS to use file system caching (i.e. FSCache).
To turn this feature on you must specifiy the -o fsc mount flag
as well as have a cachefs partition mounted. 

Signed-off-by: Steve Dickson <steved@redhat.com>


--- 2.6.12-rc5-mm2/fs/nfs/file.c.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/fs/nfs/file.c	2005-06-05 11:44:48.000000000 -0400
@@ -27,9 +27,11 @@
 #include <linux/slab.h>
 #include <linux/pagemap.h>
 #include <linux/smp_lock.h>
+#include <linux/buffer_head.h>
 
 #include <asm/uaccess.h>
 #include <asm/system.h>
+#include "nfs-fscache.h"
 
 #include "delegation.h"
 
@@ -194,6 +196,12 @@ nfs_file_sendfile(struct file *filp, lof
 	return res;
 }
 
+static int nfs_file_page_mkwrite(struct vm_area_struct *vma, struct page *page)
+{
+	wait_on_page_fs_misc(page);
+	return 0;
+}
+
 static int
 nfs_file_mmap(struct file * file, struct vm_area_struct * vma)
 {
@@ -207,6 +215,10 @@ nfs_file_mmap(struct file * file, struct
 	status = nfs_revalidate_inode(NFS_SERVER(inode), inode);
 	if (!status)
 		status = generic_file_mmap(file, vma);
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		vma->vm_ops->page_mkwrite = nfs_file_page_mkwrite;
+
 	return status;
 }
 
@@ -258,6 +270,11 @@ static int nfs_commit_write(struct file 
 	return status;
 }
 
+/*
+ * since we use page->private for our own nefarious purposes when using fscache, we have to
+ * override extra address space ops to prevent fs/buffer.c from getting confused, even though we
+ * may not have asked its opinion
+ */
 struct address_space_operations nfs_file_aops = {
 	.readpage = nfs_readpage,
 	.readpages = nfs_readpages,
@@ -269,6 +286,11 @@ struct address_space_operations nfs_file
 #ifdef CONFIG_NFS_DIRECTIO
 	.direct_IO = nfs_direct_IO,
 #endif
+#ifdef CONFIG_NFS_FSCACHE
+	.sync_page	= block_sync_page,
+	.releasepage	= nfs_releasepage,
+	.invalidatepage	= nfs_invalidatepage,
+#endif
 };
 
 /* 
--- 2.6.12-rc5-mm2/fs/nfs/inode.c.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/fs/nfs/inode.c	2005-06-05 11:44:48.000000000 -0400
@@ -42,6 +42,8 @@
 #include "nfs4_fs.h"
 #include "delegation.h"
 
+#include "nfs-fscache.h"
+
 #define NFSDBG_FACILITY		NFSDBG_VFS
 #define NFS_PARANOIA 1
 
@@ -169,6 +171,10 @@ nfs_clear_inode(struct inode *inode)
 	cred = nfsi->cache_access.cred;
 	if (cred)
 		put_rpccred(cred);
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_clear_fscookie(nfsi);
+
 	BUG_ON(atomic_read(&nfsi->data_updates) != 0);
 }
 
@@ -503,6 +509,9 @@ nfs_fill_super(struct super_block *sb, s
 			server->namelen = NFS2_MAXNAMLEN;
 	}
 
+	if (server->flags & NFS_MOUNT_FSCACHE)
+		nfs_fill_fscookie(sb);
+
 	sb->s_op = &nfs_sops;
 	return nfs_sb_init(sb, authflavor);
 }
@@ -579,6 +588,7 @@ static int nfs_show_options(struct seq_f
 		{ NFS_MOUNT_NOAC, ",noac", "" },
 		{ NFS_MOUNT_NONLM, ",nolock", ",lock" },
 		{ NFS_MOUNT_NOACL, ",noacl", "" },
+		{ NFS_MOUNT_FSCACHE, ",fscache", "" },
 		{ 0, NULL, NULL }
 	};
 	struct proc_nfs_info *nfs_infop;
@@ -623,6 +633,9 @@ nfs_zap_caches(struct inode *inode)
 		nfsi->flags |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_DATA|NFS_INO_INVALID_ACCESS|NFS_INO_INVALID_ACL;
 	else
 		nfsi->flags |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_ACCESS|NFS_INO_INVALID_ACL;
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_zap_fscookie(nfsi);
 }
 
 static void nfs_zap_acl_cache(struct inode *inode)
@@ -770,6 +783,9 @@ nfs_fhget(struct super_block *sb, struct
 		memset(nfsi->cookieverf, 0, sizeof(nfsi->cookieverf));
 		nfsi->cache_access.cred = NULL;
 
+		if (NFS_SB(sb)->flags & NFS_MOUNT_FSCACHE)
+			nfs_fhget_fscookie(sb, nfsi);
+
 		unlock_new_inode(inode);
 	} else
 		nfs_refresh_inode(inode, fattr);
@@ -1076,6 +1092,9 @@ __nfs_revalidate_inode(struct nfs_server
 				(long long)NFS_FILEID(inode));
 		/* This ensures we revalidate dentries */
 		nfsi->cache_change_attribute++;
+
+		if (server->flags & NFS_MOUNT_FSCACHE)
+			nfs_renew_fscookie(server, nfsi);
 	}
 	if (flags & NFS_INO_INVALID_ACL)
 		nfs_zap_acl_cache(inode);
@@ -1515,6 +1534,14 @@ static struct super_block *nfs_get_sb(st
 		goto out_err;
 	}
 
+#ifndef CONFIG_NFS_FSCACHE
+	if (data->flags & NFS_MOUNT_FSCACHE) {
+		printk(KERN_WARNING "NFS: kernel not compiled with CONFIG_NFS_FSCACHE\n");
+		kfree(server);
+		return ERR_PTR(-EINVAL);
+	}
+#endif
+
 	s = sget(fs_type, nfs_compare_super, nfs_set_super, server);
 	if (IS_ERR(s) || s->s_root)
 		goto out_rpciod_down;
@@ -1542,6 +1569,9 @@ static void nfs_kill_super(struct super_
 
 	kill_anon_super(s);
 
+	if (server->flags & NFS_MOUNT_FSCACHE)
+		nfs_kill_fscookie(server);
+
 	if (server->client != NULL && !IS_ERR(server->client))
 		rpc_shutdown_client(server->client);
 	if (server->client_sys != NULL && !IS_ERR(server->client_sys))
@@ -1760,6 +1790,9 @@ static int nfs4_fill_super(struct super_
 
 	sb->s_time_gran = 1;
 
+	if (server->flags & NFS4_MOUNT_FSCACHE)
+		nfs4_fill_fscookie(sb);
+
 	sb->s_op = &nfs4_sops;
 	err = nfs_sb_init(sb, authflavour);
 	if (err == 0)
@@ -1903,6 +1936,9 @@ static void nfs4_kill_super(struct super
 	nfs_return_all_delegations(sb);
 	kill_anon_super(sb);
 
+	if (server->flags & NFS_MOUNT_FSCACHE)
+		nfs_kill_fscookie(server);
+
 	nfs4_renewd_prepare_shutdown(server);
 
 	if (server->client != NULL && !IS_ERR(server->client))
@@ -2021,6 +2057,11 @@ static int __init init_nfs_fs(void)
 {
 	int err;
 
+	/* we want to be able to cache */
+	err = nfs_register_netfs();
+	if (err < 0)
+		goto out5;
+
 	err = nfs_init_nfspagecache();
 	if (err)
 		goto out4;
@@ -2068,6 +2109,9 @@ out2:
 out3:
 	nfs_destroy_nfspagecache();
 out4:
+	nfs_unregister_netfs();
+out5:
+
 	return err;
 }
 
@@ -2080,6 +2124,7 @@ static void __exit exit_nfs_fs(void)
 	nfs_destroy_readpagecache();
 	nfs_destroy_inodecache();
 	nfs_destroy_nfspagecache();
+	nfs_unregister_netfs();
 #ifdef CONFIG_PROC_FS
 	rpc_proc_unregister("nfs");
 #endif
--- 2.6.12-rc5-mm2/fs/nfs/Makefile.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/fs/nfs/Makefile	2005-06-05 11:44:48.000000000 -0400
@@ -13,4 +13,5 @@ nfs-$(CONFIG_NFS_V4)	+= nfs4proc.o nfs4x
 			   delegation.o idmap.o \
 			   callback.o callback_xdr.o callback_proc.o
 nfs-$(CONFIG_NFS_DIRECTIO) += direct.o
+nfs-$(CONFIG_NFS_FSCACHE) += nfs-fscache.o
 nfs-objs		:= $(nfs-y)
--- /dev/null	2005-06-05 03:42:13.591137792 -0400
+++ 2.6.12-rc5-mm2/fs/nfs/nfs-fscache.c	2005-06-05 11:44:48.000000000 -0400
@@ -0,0 +1,191 @@
+/* nfs-fscache.c: NFS filesystem cache interface
+ *
+ * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+
+#include <linux/config.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/nfs_fs.h>
+#include <linux/nfs_fs_sb.h>
+
+#include "nfs-fscache.h"
+
+#define NFS_CACHE_FH_INDEX_SIZE sizeof(struct nfs_fh)
+
+/*
+ * the root index is
+ */
+static struct fscache_page *nfs_cache_get_page_token(struct page *page);
+
+static struct fscache_netfs_operations nfs_cache_ops = {
+	.get_page_token	= nfs_cache_get_page_token,
+};
+
+struct fscache_netfs nfs_cache_netfs = {
+	.name			= "nfs",
+	.version		= 0,
+	.ops			= &nfs_cache_ops,
+};
+
+/*
+ * the root index for the filesystem is defined by nfsd IP address and ports
+ */
+static fscache_match_val_t nfs_cache_server_match(void *target,
+						  const void *entry);
+static void nfs_cache_server_update(void *source, void *entry);
+
+struct fscache_index_def nfs_cache_server_index_def = {
+	.name			= "servers",
+	.data_size		= 18,
+	.keys[0]		= { FSCACHE_INDEX_KEYS_IPV6ADDR, 16 },
+	.keys[1]		= { FSCACHE_INDEX_KEYS_BIN, 2 },
+	.match			= nfs_cache_server_match,
+	.update			= nfs_cache_server_update,
+};
+
+/*
+ * the primary index for each server is simply made up of a series of NFS file
+ * handles
+ */
+static fscache_match_val_t nfs_cache_fh_match(void *target, const void *entry);
+static void nfs_cache_fh_update(void *source, void *entry);
+
+struct fscache_index_def nfs_cache_fh_index_def = {
+	.name			= "fh",
+	.data_size		= NFS_CACHE_FH_INDEX_SIZE,
+	.keys[0]		= { FSCACHE_INDEX_KEYS_BIN_SZ2,
+				    sizeof(struct nfs_fh) },
+	.match			= nfs_cache_fh_match,
+	.update			= nfs_cache_fh_update,
+};
+
+/*
+ * get a page token for the specified page
+ * - the token will be attached to page->private and PG_private will be set on
+ *   the page
+ */
+static struct fscache_page *nfs_cache_get_page_token(struct page *page)
+{
+	return fscache_page_get_private(page, GFP_NOIO);
+}
+
+static const uint8_t nfs_cache_ipv6_wrapper_for_ipv4[12] = {
+	[0 ... 9]	= 0x00,
+	[10 ... 11]	= 0xff
+};
+
+/*
+ * match a server record obtained from the cache
+ */
+static fscache_match_val_t nfs_cache_server_match(void *target,
+						  const void *entry)
+{
+	struct nfs_server *server = target;
+	const uint8_t *data = entry;
+
+	switch (server->addr.sin_family) {
+	case AF_INET:
+		if (memcmp(data + 0,
+			   &nfs_cache_ipv6_wrapper_for_ipv4,
+			   12) != 0)
+			break;
+
+		if (memcmp(data + 12, &server->addr.sin_addr, 4) != 0)
+			break;
+
+		if (memcmp(data + 16, &server->addr.sin_port, 2) != 0)
+			break;
+
+		return FSCACHE_MATCH_SUCCESS;
+
+	case AF_INET6:
+		if (memcmp(data + 0, &server->addr.sin_addr, 16) != 0)
+			break;
+
+		if (memcmp(data + 16, &server->addr.sin_port, 2) != 0)
+			break;
+
+		return FSCACHE_MATCH_SUCCESS;
+
+	default:
+		break;
+	}
+
+	return FSCACHE_MATCH_FAILED;
+}
+
+/*
+ * update a server record in the cache
+ */
+static void nfs_cache_server_update(void *source, void *entry)
+{
+	struct nfs_server *server = source;
+	uint8_t *data = entry;
+
+	switch (server->addr.sin_family) {
+	case AF_INET:
+		memcpy(data + 0, &nfs_cache_ipv6_wrapper_for_ipv4, 12);
+		memcpy(data + 12, &server->addr.sin_addr, 4);
+		memcpy(data + 16, &server->addr.sin_port, 2);
+		return;
+
+	case AF_INET6:
+		memcpy(data + 0, &server->addr.sin_addr, 16);
+		memcpy(data + 16, &server->addr.sin_port, 2);
+		return;
+
+	default:
+		return;
+	}
+}
+
+/*
+ * match a file handle record obtained from the cache
+ */
+static fscache_match_val_t nfs_cache_fh_match(void *target, const void *entry)
+{
+	struct nfs_inode *nfsi = target;
+	const uint8_t *data = entry;
+	uint16_t nsize;
+
+	/* check the file handle matches */
+	memcpy(&nsize, data, 2);
+	nsize = ntohs(nsize);
+
+	if (nsize <= NFS_CACHE_FH_INDEX_SIZE && nfsi->fh.size == nsize) {
+		if (memcmp(data + 2, nfsi->fh.data, nsize) == 0) {
+			return FSCACHE_MATCH_SUCCESS;
+		}
+	}
+
+	return FSCACHE_MATCH_FAILED;
+}
+
+/*
+ * update a fh record in the cache
+ */
+static void nfs_cache_fh_update(void *source, void *entry)
+{
+	struct nfs_inode *nfsi = source;
+	uint16_t nsize;
+	uint8_t *data = entry;
+
+	BUG_ON(nfsi->fh.size > NFS_CACHE_FH_INDEX_SIZE - 2);
+
+	/* set the file handle */
+	nsize = htons(nfsi->fh.size);
+	memcpy(data, &nsize, 2);
+	memcpy(data + 2, &nfsi->fh.data, nfsi->fh.size);
+	memset(data + 2 + nfsi->fh.size,
+	       FSCACHE_INDEX_DEADFILL_PATTERN,
+	       NFS_CACHE_FH_INDEX_SIZE - 2 - nfsi->fh.size);
+}
--- /dev/null	2005-06-05 03:42:13.591137792 -0400
+++ 2.6.12-rc5-mm2/fs/nfs/nfs-fscache.h	2005-06-05 11:44:48.000000000 -0400
@@ -0,0 +1,158 @@
+/* nfs-fscache.h: NFS filesystem cache interface definitions
+ *
+ * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _NFS_FSCACHE_H
+#define _NFS_FSCACHE_H
+
+#include <linux/nfs_mount.h>
+#include <linux/nfs4_mount.h>
+#include <linux/fscache.h>
+
+#ifdef CONFIG_NFS_FSCACHE
+#ifndef CONFIG_FSCACHE
+#error "CONFIG_NFS_FSCACHE is defined but not CONFIG_FSCACHE"
+#endif
+
+extern struct fscache_netfs nfs_cache_netfs;
+extern struct fscache_index_def nfs_cache_server_index_def;
+extern struct fscache_index_def nfs_cache_fh_index_def;
+
+extern int nfs_invalidatepage(struct page *, unsigned long);
+extern int nfs_releasepage(struct page *, int);
+extern int nfs_mkwrite(struct page *);
+
+static inline void 
+nfs_renew_fscookie(struct nfs_server *server, struct nfs_inode *nfsi)
+{
+	struct fscache_cookie *old =  nfsi->fscache;
+
+	/* retire the current fscache cache and get a new one */
+	fscache_relinquish_cookie(nfsi->fscache, 1);
+	nfsi->fscache = fscache_acquire_cookie(server->fscache, NULL, nfsi);
+
+	dfprintk(FSCACHE,
+		"NFS: revalidation new cookie (0x%p/0x%p/0x%p/0x%p)\n",
+		server, nfsi, old, nfsi->fscache);
+
+	return;
+}
+static inline void
+nfs4_fill_fscookie(struct super_block *sb)
+{
+	struct nfs_server *server = NFS_SB(sb);
+
+	/* create a cache index for looking up filehandles */
+	server->fscache = fscache_acquire_cookie(nfs_cache_netfs.primary_index,
+			       &nfs_cache_fh_index_def, server);
+	if (server->fscache == NULL) {
+		printk(KERN_WARNING "NFS4: No Fscache cookie. Turning Fscache off!\n");
+	} else /* reuse the NFS mount option */
+		server->flags |= NFS_MOUNT_FSCACHE;
+
+	dfprintk(FSCACHE,"NFS: nfs4 cookie (0x%p,0x%p/0x%p)\n", 
+		sb, server, server->fscache);
+
+	return;
+}
+static inline void
+nfs_fill_fscookie(struct super_block *sb)
+{
+	struct nfs_server *server = NFS_SB(sb);
+
+	/* create a cache index for looking up filehandles */
+	server->fscache = fscache_acquire_cookie(nfs_cache_netfs.primary_index,
+			      &nfs_cache_fh_index_def, server);
+	if (server->fscache == NULL) {
+		server->flags &= ~NFS_MOUNT_FSCACHE;
+		printk(KERN_WARNING "NFS: No Fscache cookie. Turning Fscache off!\n");
+	}
+	dfprintk(FSCACHE,"NFS: cookie (0x%p/0x%p/0x%p)\n", 
+		sb, server, server->fscache);
+
+	return;
+}
+static inline void
+nfs_fhget_fscookie(struct super_block *sb, struct nfs_inode *nfsi)
+{
+	struct nfs_server *server = NFS_SB(sb);
+
+	nfsi->fscache = fscache_acquire_cookie(server->fscache, NULL, nfsi);
+	if (server->fscache == NULL)
+		printk(KERN_WARNING "NFS: NULL FScache cookie: sb 0x%p nfsi 0x%p\n", sb, nfsi);
+
+	dfprintk(FSCACHE, "NFS: fhget new cookie (0x%p/0x%p/0x%p)\n",
+		sb, nfsi, nfsi->fscache);
+
+	return;
+}
+static inline void 
+nfs_kill_fscookie(struct nfs_server *server)
+{
+	dfprintk(FSCACHE,"NFS: killing cookie (0x%p/0x%p)\n",
+		server, server->fscache);
+
+	fscache_relinquish_cookie(server->fscache, 0);
+	server->fscache = NULL;
+
+	return;
+}
+static inline void
+nfs_clear_fscookie(struct nfs_inode *nfsi)
+{
+	dfprintk(FSCACHE, "NFS: clear cookie (0x%p/0x%p)\n",
+			nfsi, nfsi->fscache);
+
+	fscache_relinquish_cookie(nfsi->fscache, 0);
+	nfsi->fscache = NULL;
+
+	return;
+}
+static inline void
+nfs_zap_fscookie(struct nfs_inode *nfsi)
+{
+	dfprintk(FSCACHE,"NFS: zapping cookie (0x%p/0x%p)\n",
+		nfsi, nfsi->fscache);
+
+	fscache_relinquish_cookie(nfsi->fscache, 1);
+	nfsi->fscache = NULL;
+
+	return;
+}
+static inline int
+nfs_register_netfs(void)
+{
+	int err;
+
+	err = fscache_register_netfs(&nfs_cache_netfs, &nfs_cache_server_index_def);
+
+	return err;
+}
+static inline void
+nfs_unregister_netfs(void)
+{
+	fscache_unregister_netfs(&nfs_cache_netfs);
+
+	return;
+}
+#else
+static inline void nfs_fill_fscookie(struct super_block *sb) {}
+static inline void nfs_fhget_fscookie(struct super_block *sb, struct nfs_inode *nfsi) {}
+static inline void nfs4_fill_fscookie(struct super_block *sb) {}
+static inline void nfs_kill_fscookie(struct nfs_server *server) {}
+static inline void nfs_clear_fscookie(struct nfs_inode *nfsi) {}
+static inline void nfs_zap_fscookie(struct nfs_inode *nfsi) {}
+static inline void 
+	nfs_renew_fscookie(struct nfs_server *server, struct nfs_inode *nfsi) {}
+static inline int nfs_register_netfs() { return 0; }
+static inline void nfs_unregister_netfs() {}
+
+#endif
+#endif /* _NFS_FSCACHE_H */
--- 2.6.12-rc5-mm2/fs/nfs/read.c.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/fs/nfs/read.c	2005-06-05 11:44:48.000000000 -0400
@@ -27,6 +27,7 @@
 #include <linux/sunrpc/clnt.h>
 #include <linux/nfs_fs.h>
 #include <linux/nfs_page.h>
+#include <linux/nfs_mount.h>
 #include <linux/smp_lock.h>
 
 #include <asm/system.h>
@@ -73,6 +74,47 @@ int nfs_return_empty_page(struct page *p
 	return 0;
 }
 
+#ifdef CONFIG_NFS_FSCACHE
+/*
+ * store a newly fetched page in fscache
+ */
+static void
+nfs_readpage_to_fscache_complete(void *cookie_data, struct page *page, void *data, int error)
+{
+	dprintk("NFS:     readpage_to_fscache_complete (%p/%p/%p/%d)\n",
+		cookie_data, page, data, error);
+
+	end_page_fs_misc(page);
+}
+
+static inline void
+nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)
+{
+	int ret;
+
+	dprintk("NFS: readpage_to_fscache(0x%p/0x%p/0x%p/%d)\n",
+		NFS_I(inode)->fscache, page, inode, sync);
+
+	SetPageFsMisc(page);
+	ret = fscache_write_page(NFS_I(inode)->fscache, page,
+		nfs_readpage_to_fscache_complete, NULL, GFP_KERNEL);
+	if (ret != 0) {
+		dprintk("NFS:     readpage_to_fscache: error %d\n", ret);
+		fscache_uncache_page(NFS_I(inode)->fscache, page);
+		ClearPageFsMisc(page);
+	}
+
+	unlock_page(page);
+}
+#else
+static inline void
+nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)
+{
+	BUG();
+}
+#endif
+
+
 /*
  * Read a page synchronously.
  */
@@ -149,6 +191,13 @@ static int nfs_readpage_sync(struct nfs_
 		ClearPageError(page);
 	result = 0;
 
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_readpage_to_fscache(inode, page, 1);
+	else
+		unlock_page(page);
+
+	return result;
+
 io_error:
 	unlock_page(page);
 	nfs_readdata_free(rdata);
@@ -180,7 +229,13 @@ static int nfs_readpage_async(struct nfs
 
 static void nfs_readpage_release(struct nfs_page *req)
 {
-	unlock_page(req->wb_page);
+	struct inode *d_inode = req->wb_context->dentry->d_inode;
+
+	if ((NFS_SERVER(d_inode)->flags & NFS_MOUNT_FSCACHE) && 
+			PageUptodate(req->wb_page))
+		nfs_readpage_to_fscache(d_inode, req->wb_page, 0);
+	else
+		unlock_page(req->wb_page);
 
 	nfs_clear_request(req);
 	nfs_release_request(req);
@@ -477,6 +532,67 @@ void nfs_readpage_result(struct rpc_task
 	data->complete(data, status);
 }
 
+
+/*
+ * Read a page through the on-disc cache if possible
+ */
+#ifdef CONFIG_NFS_FSCACHE
+static void
+nfs_readpage_from_fscache_complete(void *cookie_data, struct page *page, void *data, int error)
+{
+	dprintk("NFS: readpage_from_fscache_complete (0x%p/0x%p/0x%p/%d)\n",
+		cookie_data, page, data, error);
+
+	if (error)
+		SetPageError(page);
+	else
+		SetPageUptodate(page);
+
+	unlock_page(page);
+}
+
+static inline int
+nfs_readpage_from_fscache(struct inode *inode, struct page *page)
+{
+	struct fscache_page *pageio;
+	int ret;
+
+	dprintk("NFS: readpage_from_fscache(0x%p/0x%p/0x%p)\n",
+		NFS_I(inode)->fscache, page, inode);
+
+	pageio = fscache_page_get_private(page, GFP_NOIO);
+	if (IS_ERR(pageio)) {
+		 dprintk("NFS:     fscache_page_get_private error %ld\n", PTR_ERR(pageio));
+		return PTR_ERR(pageio);
+	}
+
+	ret = fscache_read_or_alloc_page(NFS_I(inode)->fscache,
+					 page,
+					 nfs_readpage_from_fscache_complete,
+					 NULL,
+					 GFP_KERNEL);
+
+	switch (ret) {
+	case 1: /* read BIO submitted and wb-journal entry found */
+		BUG();
+		
+	case 0: /* read BIO submitted (page in fscache) */
+		return ret;
+
+	case -ENOBUFS: /* inode not in cache */
+	case -ENODATA: /* page not in cache */
+		dprintk("NFS:     fscache_read_or_alloc_page error %d\n", ret);
+		return 1;
+
+	default:
+		return ret;
+	}
+}
+#else
+static inline int 
+nfs_readpage_from_fscache(struct inode *inode, struct page *page) { return 1; }
+#endif
+
 /*
  * Read a page over NFS.
  * We read the page synchronously in the following case:
@@ -510,6 +626,13 @@ int nfs_readpage(struct file *file, stru
 		ctx = get_nfs_open_context((struct nfs_open_context *)
 				file->private_data);
 	if (!IS_SYNC(inode)) {
+		if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE) {
+			error = nfs_readpage_from_fscache(inode, page);
+			if (error < 0)
+				goto out_error;
+			if (error == 0)
+				return error;
+		}
 		error = nfs_readpage_async(ctx, inode, page);
 		goto out;
 	}
@@ -540,6 +663,15 @@ readpage_async_filler(void *data, struct
 	unsigned int len;
 
 	nfs_wb_page(inode, page);
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE) {
+		int error = nfs_readpage_from_fscache(inode, page);
+		if (error < 0)
+			return error;
+		if (error == 0)
+			return error;
+	}
+
 	len = nfs_page_length(inode, page);
 	if (len == 0)
 		return nfs_return_empty_page(page);
@@ -613,3 +745,61 @@ void nfs_destroy_readpagecache(void)
 	if (kmem_cache_destroy(nfs_rdata_cachep))
 		printk(KERN_INFO "nfs_read_data: not all structures were freed\n");
 }
+
+#ifdef CONFIG_NFS_FSCACHE
+int nfs_invalidatepage(struct page *page, unsigned long offset)
+{
+	int ret = 1;
+	struct nfs_server *server = NFS_SERVER(page->mapping->host);
+
+	BUG_ON(!PageLocked(page));
+
+	if (server->flags & NFS_MOUNT_FSCACHE) {
+		if (PagePrivate(page)) {
+			struct nfs_inode *nfsi = NFS_I(page->mapping->host);
+
+			dfprintk(PAGECACHE,"NFS: fscache invalidatepage (0x%p/0x%p/0x%p)\n",
+				nfsi->fscache, page, nfsi);
+
+			fscache_uncache_page(nfsi->fscache, page);
+
+			if (offset == 0) {
+				BUG_ON(!PageLocked(page));
+				ret = 0;
+				if (!PageWriteback(page))
+					ret = page->mapping->a_ops->releasepage(page, 0);
+			}
+		}
+	} else
+		ret = 0;
+
+	return ret;
+}
+int nfs_releasepage(struct page *page, int gfp_flags)
+{
+	struct fscache_page *pageio;
+	struct nfs_server *server = NFS_SERVER(page->mapping->host);
+
+	if (server->flags & NFS_MOUNT_FSCACHE && PagePrivate(page)) {
+		struct nfs_inode *nfsi = NFS_I(page->mapping->host);
+
+		dfprintk(PAGECACHE,"NFS: fscache releasepage (0x%p/0x%p/0x%p)\n",
+			nfsi->fscache, page, nfsi);
+
+		fscache_uncache_page(nfsi->fscache, page);
+		pageio = (struct fscache_page *) page->private;
+		page->private = 0;
+		ClearPagePrivate(page);
+
+		if (pageio)
+			kfree(pageio);
+	}
+
+	return 0;
+}
+int nfs_mkwrite(struct page *page)
+{
+	wait_on_page_fs_misc(page);
+	return 0;
+}
+#endif
--- 2.6.12-rc5-mm2/fs/nfs/write.c.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/fs/nfs/write.c	2005-06-05 11:44:48.000000000 -0400
@@ -255,6 +255,38 @@ static int wb_priority(struct writeback_
 }
 
 /*
+ * store an updated page in fscache
+ */
+#ifdef CONFIG_NFS_FSCACHE
+static void
+nfs_writepage_to_fscache_complete(void *cookie_data, struct page *page, void *data, int error)
+{
+	/* really need to synchronise the end of writeback, probably using a page flag */
+}
+static inline void
+nfs_writepage_to_fscache(struct inode *inode, struct page *page)
+{
+	int ret; 
+
+	dprintk("NFS: writepage_to_fscache (0x%p/0x%p/0x%p)\n",
+		NFS_I(inode)->fscache, page, inode);
+
+	ret =  fscache_write_page(NFS_I(inode)->fscache, page,
+		nfs_writepage_to_fscache_complete, NULL, GFP_KERNEL);
+	if (ret != 0) {
+		dprintk("NFS:    fscache_write_page error %d\n", ret);
+		fscache_uncache_page(NFS_I(inode)->fscache, page);
+	}
+}
+#else
+static inline void
+nfs_writepage_to_fscache(struct inode *inode, struct page *page)
+{
+	BUG();
+}
+#endif
+
+/*
  * Write an mmapped page to the server.
  */
 int nfs_writepage(struct page *page, struct writeback_control *wbc)
@@ -299,6 +331,10 @@ do_it:
 		err = -EBADF;
 		goto out;
 	}
+
+	if (NFS_SERVER(inode)->flags & NFS_MOUNT_FSCACHE)
+		nfs_writepage_to_fscache(inode, page);
+
 	lock_kernel();
 	if (!IS_SYNC(inode) && inode_referenced) {
 		err = nfs_writepage_async(ctx, inode, page, 0, offset);
--- 2.6.12-rc5-mm2/fs/Kconfig.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/fs/Kconfig	2005-06-05 11:44:48.000000000 -0400
@@ -1495,6 +1495,13 @@ config NFS_V4
 
 	  If unsure, say N.
 
+config NFS_FSCACHE
+	bool "Provide NFS client caching support (EXPERIMENTAL)"
+	depends on NFS_FS && FSCACHE && EXPERIMENTAL
+	help
+	  Say Y here if you want NFS data to be cached locally on disc through
+	  the general filesystem cache manager
+
 config NFS_DIRECTIO
 	bool "Allow direct I/O on NFS files (EXPERIMENTAL)"
 	depends on NFS_FS && EXPERIMENTAL
--- 2.6.12-rc5-mm2/include/linux/nfs_fs.h.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/include/linux/nfs_fs.h	2005-06-05 11:44:48.000000000 -0400
@@ -29,6 +29,7 @@
 #include <linux/nfs_xdr.h>
 #include <linux/rwsem.h>
 #include <linux/mempool.h>
+#include <linux/fscache.h>
 
 /*
  * Enable debugging support for nfs client.
@@ -184,6 +185,11 @@ struct nfs_inode {
 	int			 delegation_state;
 	struct rw_semaphore	rwsem;
 #endif /* CONFIG_NFS_V4*/
+
+#ifdef CONFIG_NFS_FSCACHE
+	struct fscache_cookie	*fscache;
+#endif
+
 	struct inode		vfs_inode;
 };
 
@@ -564,6 +570,7 @@ extern void * nfs_root_data(void);
 #define NFSDBG_FILE		0x0040
 #define NFSDBG_ROOT		0x0080
 #define NFSDBG_CALLBACK		0x0100
+#define NFSDBG_FSCACHE		0x0200
 #define NFSDBG_ALL		0xFFFF
 
 #ifdef __KERNEL__
--- 2.6.12-rc5-mm2/include/linux/nfs_fs_sb.h.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/include/linux/nfs_fs_sb.h	2005-06-05 11:44:48.000000000 -0400
@@ -3,6 +3,7 @@
 
 #include <linux/list.h>
 #include <linux/backing-dev.h>
+#include <linux/fscache.h>
 
 /*
  * NFS client parameters stored in the superblock.
@@ -47,6 +48,10 @@ struct nfs_server {
 						   that are supported on this
 						   filesystem */
 #endif
+
+#ifdef CONFIG_NFS_FSCACHE
+	struct fscache_cookie	*fscache;	/* cache cookie */
+#endif
 };
 
 /* Server capabilities */
--- 2.6.12-rc5-mm2/include/linux/nfs_mount.h.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/include/linux/nfs_mount.h	2005-06-05 11:44:48.000000000 -0400
@@ -61,6 +61,7 @@ struct nfs_mount_data {
 #define NFS_MOUNT_NOACL		0x0800	/* 4 */
 #define NFS_MOUNT_STRICTLOCK	0x1000	/* reserved for NFSv4 */
 #define NFS_MOUNT_SECFLAVOUR	0x2000	/* 5 */
+#define NFS_MOUNT_FSCACHE		0x3000
 #define NFS_MOUNT_FLAGMASK	0xFFFF
 
 #endif
--- 2.6.12-rc5-mm2/include/linux/nfs4_mount.h.orig	2005-06-05 11:44:35.000000000 -0400
+++ 2.6.12-rc5-mm2/include/linux/nfs4_mount.h	2005-06-05 11:44:48.000000000 -0400
@@ -65,6 +65,7 @@ struct nfs4_mount_data {
 #define NFS4_MOUNT_NOCTO	0x0010	/* 1 */
 #define NFS4_MOUNT_NOAC		0x0020	/* 1 */
 #define NFS4_MOUNT_STRICTLOCK	0x1000	/* 1 */
+#define NFS4_MOUNT_FSCACHE	0x2000	/* 1 */
 #define NFS4_MOUNT_FLAGMASK	0xFFFF
 
 #endif

[-- Attachment #3: 2.6.12-rc6-mm1-fscache-cookie-exist.patch --]
[-- Type: text/x-patch, Size: 669 bytes --]

Fails a second NFS mount with EEXIST instead of oops. 

Signed-off-by: Steve Dickson <steved@redhat.com>

--- 2.6.12-rc3-mm3/fs/fscache/cookie.c.orig	2005-05-07 09:30:28.000000000 -0400
+++ 2.6.12-rc3-mm3/fs/fscache/cookie.c	2005-05-07 11:01:39.000000000 -0400
@@ -452,7 +452,11 @@ static int fscache_search_for_object(str
 		cache->ops->lock_node(node);
 
 		/* a node should only ever be attached to one cookie */
-		BUG_ON(!list_empty(&node->cookie_link));
+		if (!list_empty(&node->cookie_link)) {
+			cache->ops->unlock_node(node);
+			ret = -EEXIST;
+			goto error;
+		}
 
 		/* attach the node to the cache's node list */
 		if (list_empty(&node->cache_link)) {

[-- Attachment #4: 2.6.12-rc6-mm1-cachefs-wb.patch --]
[-- Type: text/x-patch, Size: 594 bytes --]

This fixes a BUG() poping at mm/filemap.c:465 pops when reading a 
100m file using nfs4. 

Signed-off-by: Steve Dickson <steved@redhat.com>


--- 2.6.12-rc2-mm3/fs/cachefs/journal.c.save	2005-04-27 08:06:03.000000000 -0400
+++ 2.6.12-rc2-mm3/fs/cachefs/journal.c	2005-05-03 11:11:17.000000000 -0400
@@ -682,6 +682,7 @@ static inline void cachefs_trans_batch_p
 		list_add_tail(&block->batch_link, plist);
 		block->writeback = block->page;
 		get_page(block->writeback);
+		SetPageWriteback(block->writeback);
 
 		/* make sure DMA can reach the data */
 		flush_dcache_page(block->writeback);

[-- Attachment #5: Type: text/plain, Size: 0 bytes --]