NFS and non-blkdev-backed FS?

NFS and non-blkdev-backed FS?

[Artem Bityutskiy]

When I NFS-export UBIFS, I see the following warning:
exportfs: Warning: /mnt/ubifs requires fsid= for NFS export

And the client won't mount UBIFS:
mount.nfs: access denied by server while mounting gollum:/mnt/ubifs

The problem is solved by adding 'fsid=5' to /etc/exports. And BTW,
exactly the same happens to JFFS2.

This looks strange to me, probably because I misunderstand
something.

AFAIU, NFS uses major/minor numbers of block device to create
FSID and uses it as part of file handle. And the idea behind
this is that if the server rebooted unexpectedly, and then
exported a _different_ file-system to the same path, the client
would notice this because the FSID part of file handle would
change.

Well, I do not see how it solves the problem if the hard drive has
been changed, though.

Since UBIFS (and JFFS2) does not work on top of block devices, NFS
cannot compose FSID from major/minor numbers and exportfs yells.

But I do not like the idea of adding explicit fsid=5 to /etc/exports,
because it does not seem to solve the above problem, surely?

UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is
supposed to uniquely identify the file-system. I'd like to use
this UUID as NFS FSID instead, however, I failed to find how
to do this. Any hints how could I do this?

Thanks.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: NFS and non-blkdev-backed FS?

[Christoph Hellwig]

On Fri, Aug 15, 2008 at 11:55:59AM +0300, Artem Bityutskiy wrote:
> UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is
> supposed to uniquely identify the file-system. I'd like to use
> this UUID as NFS FSID instead, however, I failed to find how
> to do this. Any hints how could I do this?

You would need a new export operations method for initializating the
the fsid part of the file handle.  This method would be calked from
fh_compose or rather the new fsid generating factored out from that.

Note that it would be good to use this for block bases filesystems, too
so there should be some way to override the default fsids for those,
too.

Re: NFS and non-blkdev-backed FS?

[David Woodhouse]

On Fri, 2008-08-15 at 11:55 +0300, Artem Bityutskiy wrote:
> UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is
> supposed to uniquely identify the file-system. I'd like to use
> this UUID as NFS FSID instead, however, I failed to find how
> to do this. Any hints how could I do this?

By adding a 'get_fsid()' method to the export_ops structure...

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: NFS and non-blkdev-backed FS?

[Trond Myklebust]

On Fri, 2008-08-15 at 14:59 +0100, David Woodhouse wrote:
> On Fri, 2008-08-15 at 11:55 +0300, Artem Bityutskiy wrote:
> > UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is
> > supposed to uniquely identify the file-system. I'd like to use
> > this UUID as NFS FSID instead, however, I failed to find how
> > to do this. Any hints how could I do this?
> 
> By adding a 'get_fsid()' method to the export_ops structure...
> 

If the uuid can be found by libuuid, then it will automatically be used
as the default identifier by rpc.mount (man 5 exports). There should be
no need for a new export_ops method.

Cheers
  Trond

Re: NFS and non-blkdev-backed FS?

[Christoph Hellwig]

On Fri, Aug 15, 2008 at 10:18:12AM -0400, Trond Myklebust wrote:
> On Fri, 2008-08-15 at 14:59 +0100, David Woodhouse wrote:
> > On Fri, 2008-08-15 at 11:55 +0300, Artem Bityutskiy wrote:
> > > UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is
> > > supposed to uniquely identify the file-system. I'd like to use
> > > this UUID as NFS FSID instead, however, I failed to find how
> > > to do this. Any hints how could I do this?
> > 
> > By adding a 'get_fsid()' method to the export_ops structure...
> > 
> 
> If the uuid can be found by libuuid, then it will automatically be used
> as the default identifier by rpc.mount (man 5 exports). There should be
> no need for a new export_ops method.

We'll need it anyway, for exports not starting at the root of the
volume, and we'll need a method to be called when exporting a filesystem
to reject it if the filehandle couldn't fit into the space available
for the given nfs protocol version.

Re: NFS and non-blkdev-backed FS?

[Trond Myklebust]

On Sat, 2008-08-16 at 14:11 -0400, Christoph Hellwig wrote:
> On Fri, Aug 15, 2008 at 10:18:12AM -0400, Trond Myklebust wrote:
> > On Fri, 2008-08-15 at 14:59 +0100, David Woodhouse wrote:
> > > On Fri, 2008-08-15 at 11:55 +0300, Artem Bityutskiy wrote:
> > > > UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is
> > > > supposed to uniquely identify the file-system. I'd like to use
> > > > this UUID as NFS FSID instead, however, I failed to find how
> > > > to do this. Any hints how could I do this?
> > > 
> > > By adding a 'get_fsid()' method to the export_ops structure...
> > > 
> > 
> > If the uuid can be found by libuuid, then it will automatically be used
> > as the default identifier by rpc.mount (man 5 exports). There should be
> > no need for a new export_ops method.
> 
> We'll need it anyway, for exports not starting at the root of the
> volume, and we'll need a method to be called when exporting a filesystem
> to reject it if the filehandle couldn't fit into the space available
> for the given nfs protocol version.

So far we've been using the pair (fsid:fileid) to label the export
inode. Why does using a UUID as the fsid change that?

As for the problem of dealing with a uuid-based filehandle not fitting
into the NFSv2 maximum of 32-bytes, why is that any different from the
general problem of defining alternative globally unique identifiers, and
why do we now think we can do that better in the kernel rather than
relying on the previous strategy of defining such identifiers in
userland?

Re: NFS and non-blkdev-backed FS?

[David Woodhouse]

On Fri, 2008-08-15 at 10:18 -0400, Trond Myklebust wrote:
> On Fri, 2008-08-15 at 14:59 +0100, David Woodhouse wrote:
> > On Fri, 2008-08-15 at 11:55 +0300, Artem Bityutskiy wrote:
> > > UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is
> > > supposed to uniquely identify the file-system. I'd like to use
> > > this UUID as NFS FSID instead, however, I failed to find how
> > > to do this. Any hints how could I do this?
> > 
> > By adding a 'get_fsid()' method to the export_ops structure...

That approach is painful -- it's simple enough to make the export work,
and it returns an appropriate root fh -- but then falls over immediately
when the client gives that fh back to us in an FSINFO request and
userspace mountd doesn't understand it. A hack to prepopulate the
svc_export_cache too 'fixes' it, but won't survive reboots.

> If the uuid can be found by libuuid, then it will automatically be used
> as the default identifier by rpc.mount (man 5 exports). There should be
> no need for a new export_ops method.

The problem with libuuid is that it assumes that there's a block device,
and a 1:1 mapping between file system and block device. That often isn't
true.

However, there _is_ a f_fsid field in struct statfs64, which we could
happily use if libuuid fails.

See git.infradead.org/users/dwmw2/nfs-utils.git -- if the file system
fills in f_fsid, that should be all that's needed.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

[PATCH 1/3] Fix handling of explicit uuid

[David Woodhouse]

Fix a couple of bugs which show up if you try to explicitly set a
16-byte UUID when exporting a file system. First, exportfs cuts the
first two bytes off the UUID and writes something invalid to etab.
Second, mountd writes the _ascii_ form of the UUID to the kernel,
instead of converting it to hex.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
 support/nfs/exports.c |    2 +-
 utils/mountd/cache.c  |    4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/support/nfs/exports.c b/support/nfs/exports.c
index 525e5b1..334c08e 100644
--- a/support/nfs/exports.c
+++ b/support/nfs/exports.c
@@ -595,7 +595,7 @@ bad_option:
 				if (opt[5]!='\0' && *oe == '\0') 
 					ep->e_flags |= NFSEXP_FSID;
 				else if (valid_uuid(opt+5))
-					ep->e_uuid = strdup(opt+7);
+					ep->e_uuid = strdup(opt+5);
 				else {
 					xlog(L_ERROR, "%s: %d: bad fsid \"%s\"\n",
 					     flname, flline, opt);	
diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
index f555dcc..cd377ce 100644
--- a/utils/mountd/cache.c
+++ b/utils/mountd/cache.c
@@ -590,8 +590,10 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
  				qword_printhex(f, u, 16);
  			}
  		} else {
+			char u[16];
+			get_uuid(NULL, exp->e_uuid, 16, u);
  			qword_print(f, "uuid");
- 			qword_printhex(f, exp->e_uuid, 16);
+ 			qword_printhex(f, u, 16);
  		}
 #endif
 	}
-- 
1.5.5.1


-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

[PATCH 2/3] Explicit UUID handling doesn't require blkid; factor out get_uuid_blkdev()

[David Woodhouse]

Signed-off-by: David Woodhouse <David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 utils/mountd/cache.c |   74 +++++++++++++++++++++++++++-----------------------
 1 files changed, 40 insertions(+), 34 deletions(-)

diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
index cd377ce..0136eca 100644
--- a/utils/mountd/cache.c
+++ b/utils/mountd/cache.c
@@ -14,6 +14,7 @@
 #include <sys/types.h>
 #include <sys/select.h>
 #include <sys/stat.h>
+#include <sys/vfs.h>
 #include <time.h>
 #include <netinet/in.h>
 #include <arpa/inet.h>
@@ -165,6 +166,41 @@ void auth_unix_gid(FILE *f)
 }
 
 #if USE_BLKID
+static const char *get_uuid_blkdev(char *path)
+{
+	static blkid_cache cache = NULL;
+	struct stat stb;
+	char *devname;
+	blkid_tag_iterate iter;
+	blkid_dev dev;
+	const char *type;
+	const char *val = NULL;
+
+	if (cache == NULL)
+		blkid_get_cache(&cache, NULL);
+
+	if (stat(path, &stb) != 0)
+		return NULL;
+	devname = blkid_devno_to_devname(stb.st_dev);
+	if (!devname)
+		return NULL;
+	dev = blkid_get_dev(cache, devname, BLKID_DEV_NORMAL);
+	free(devname);
+	if (!dev)
+		return NULL;
+	iter = blkid_tag_iterate_begin(dev);
+	if (!iter)
+		return NULL;
+	while (blkid_tag_next(iter, &type, &val) == 0)
+		if (strcmp(type, "UUID") == 0)
+			break;
+	blkid_tag_iterate_end(iter);
+	return val;
+}
+#else
+#define get_uuid_blkdev(path) (NULL)
+#endif
+
 int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 {
 	/* extract hex digits from uuidstr and compose a uuid
@@ -172,35 +208,12 @@ int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 	 * a smaller uuid.  Then compare with uuid
 	 */
 	int i = 0;
-	const char *val;
+	const char *val = NULL;
+	char fsid_val[17];
 
 	if (path) {
-		static blkid_cache cache = NULL;
-		struct stat stb;
-		char *devname;
-		blkid_tag_iterate iter;
-		blkid_dev dev;
-		const char *type;
-		if (cache == NULL)
-			blkid_get_cache(&cache, NULL);
-
-		if (stat(path, &stb) != 0)
-			return 0;
-		devname = blkid_devno_to_devname(stb.st_dev);
-		if (!devname)
-			return 0;
-		dev = blkid_get_dev(cache, devname, BLKID_DEV_NORMAL);
-		free(devname);
-		if (!dev)
-			return 0;
-		iter = blkid_tag_iterate_begin(dev);
-		if (!iter)
-			return 0;
-		while (blkid_tag_next(iter, &type, &val) == 0)
-			if (strcmp(type, "UUID") == 0)
-				break;
-		blkid_tag_iterate_end(iter);
-		if (!type)
+		val = get_uuid_blkdev(path);
+		if (!val)
 			return 0;
 	} else {
 		val = uuid;
@@ -227,7 +240,6 @@ int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 	}
 	return 1;
 }
-#endif
 
 /* Iterate through /etc/mtab, finding mountpoints
  * at or below a given path
@@ -437,7 +449,6 @@ void nfsd_fh(FILE *f)
 				if (!is_mountpoint(path))
 					continue;
 			check_uuid:
-#if USE_BLKID
 				if (exp->m_export.e_uuid)
 					get_uuid(NULL, exp->m_export.e_uuid,
 						 uuidlen, u);
@@ -448,9 +459,6 @@ void nfsd_fh(FILE *f)
 				if (memcmp(u, fhuuid, uuidlen) != 0)
 					continue;
 				break;
-#else
-				continue;
-#endif
 			}
 			if (use_ipaddr) {
 				if (he == NULL) {
@@ -582,7 +590,6 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
 		qword_printint(f, exp->e_fsid);
 		write_fsloc(f, exp, path);
 		write_secinfo(f, exp);
-#if USE_BLKID
  		if (exp->e_uuid == NULL || different_fs) {
  			char u[16];
  			if (get_uuid(path, NULL, 16, u)) {
@@ -595,7 +602,6 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
  			qword_print(f, "uuid");
  			qword_printhex(f, u, 16);
  		}
-#endif
 	}
 	return qword_eol(f);
 }
-- 
1.5.5.1


-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org                              Intel Corporation



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] Use fsid from statfs for UUID if blkid can't cope (or not used)

[David Woodhouse]

Signed-off-by: David Woodhouse <David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 utils/mountd/cache.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
index 0136eca..f29b885 100644
--- a/utils/mountd/cache.c
+++ b/utils/mountd/cache.c
@@ -213,8 +213,17 @@ int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 
 	if (path) {
 		val = get_uuid_blkdev(path);
-		if (!val)
-			return 0;
+		if (!val) {
+			struct statfs64 st;
+
+			if (statfs64(path, &st))
+				return 0;
+			if (!st.f_fsid.__val[0] && !st.f_fsid.__val[1])
+				return 0;
+			snprintf(fsid_val, 17, "%08x%08x",
+				 st.f_fsid.__val[0], st.f_fsid.__val[1]);
+			val = fsid_val;
+		}
 	} else {
 		val = uuid;
 	}
-- 
1.5.5.1


-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org                              Intel Corporation



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/3] Stop exportfs warning about needing fsid, when we actually have one

[David Woodhouse]

Signed-off-by: David Woodhouse <David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 utils/exportfs/exportfs.c |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/utils/exportfs/exportfs.c b/utils/exportfs/exportfs.c
index 18dfe5a..fec2571 100644
--- a/utils/exportfs/exportfs.c
+++ b/utils/exportfs/exportfs.c
@@ -12,6 +12,7 @@
 #include <config.h>
 #endif
 
+#include <sys/vfs.h>
 #include <unistd.h>
 #include <stdlib.h>
 #include <string.h>
@@ -388,6 +389,8 @@ validate_export(nfs_export *exp)
 	 */
 	struct stat stb;
 	char *path = exp->m_export.e_path;
+	struct statfs64 stf;
+	int fs_has_fsid = 0;
 
 	if (stat(path, &stb) < 0) {
 		fprintf(stderr, "exportfs: Warning: %s does not exist\n",
@@ -403,7 +406,12 @@ validate_export(nfs_export *exp)
 	if (!can_test())
 		return;
 
-	if ((exp->m_export.e_flags & NFSEXP_FSID) || exp->m_export.e_uuid) {
+	if (!statfs64(path, &stf) &&
+	    (stf.f_fsid.__val[0] || stf.f_fsid.__val[1]))
+		fs_has_fsid = 1;
+
+	if ((exp->m_export.e_flags & NFSEXP_FSID) || exp->m_export.e_uuid ||
+	    fs_has_fsid) {
 		if ( !test_export(path, 1)) {
 			fprintf(stderr, "exportfs: Warning: %s does not "
 				"support NFS export.\n",
-- 
1.5.5.1



-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org                              Intel Corporation



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: NFS and non-blkdev-backed FS?

[Bryan Henderson]

>AFAIU, NFS uses major/minor numbers of block device to create
>FSID and uses it as part of file handle. And the idea behind
>this is that if the server rebooted unexpectedly, and then
>exported a _different_ file-system to the same path, the client
>would notice this because the FSID part of file handle would
>change.

After the reboot, the client would not notice that that the server had 
exported a different filesystem to the same path because the client has 
already done the lookup; he has no reason to do it again.  (If the 
original file is still exported under a different name, the client will 
continue accessing the old file.  If the original filesystem isn't 
exported at all anymore, the client's access via the old file handle fails 
and the client repeats the lookup and continues accessing the new file as 
if nothing had changed).

The point of the FSID is simply that the file handle has to completely 
identify the file so the server knows which file to access.  An NFS client 
request to read a file doesn't come with a file name; the file handle is 
all there is.

If the filesystem lives on a disk device and you trust the disk device's 
major/minor number not to change across a reboot, major/minor number makes 
a good FSID.  If not, you need something else, such as the filesystem UUID 
you propose.

Having the system administrator manually identify the filesystem in a 
special namespace via fsid=5 in /etc/exports obviously doesn't have the 
same level of integrity, but it does solve the problem of not having any 
way to identify the filesystem at all.

Re: [PATCH 1/3] Fix handling of explicit uuid

[Steve Dickson]

David Woodhouse wrote:
> Fix a couple of bugs which show up if you try to explicitly set a
> 16-byte UUID when exporting a file system. First, exportfs cuts the
> first two bytes off the UUID and writes something invalid to etab.
> Second, mountd writes the _ascii_ form of the UUID to the kernel,
> instead of converting it to hex.
> 
> Signed-off-by: David Woodhouse <David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Committed...

steved
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/3] Explicit UUID handling doesn't require blkid; factor out get_uuid_blkdev()

[Steve Dickson]

David Woodhouse wrote:
> Signed-off-by: David Woodhouse <David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Committed...

steved.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] Use fsid from statfs for UUID if blkid can't cope (or not used)

[Steve Dickson]

David Woodhouse wrote:
> Signed-off-by: David Woodhouse <David.Woodhouse-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Committed...

steved.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/3] Stop exportfs warning about needing fsid, when we actually have one

[Steve Dickson]

David Woodhouse wrote:
> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>

Committed...

steved.