* [PATCH 01/14] ovl: hash all overlay inodes for NFS export
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 02/14] ovl: grab i_count reference of lower inode Amir Goldstein
` (14 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
If NFS export is enabled, hash all overlay inodes, so we can find
them in inode cache using the decoded real inode as the hash key.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/inode.c | 8 ++++++--
fs/overlayfs/super.c | 3 +++
fs/overlayfs/util.c | 4 +++-
3 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 321511ed8c42..a792c088c2b4 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -524,7 +524,9 @@ unsigned int ovl_get_nlink(struct dentry *lowerdentry,
char buf[13];
int err;
- if (!lowerdentry || !upperdentry || d_inode(lowerdentry)->i_nlink == 1)
+ if (!lowerdentry || !upperdentry ||
+ d_inode(lowerdentry)->i_nlink == 1 ||
+ S_ISDIR(d_inode(upperdentry)->i_mode))
return fallback;
err = vfs_getxattr(upperdentry, OVL_XATTR_NLINK, &buf, sizeof(buf) - 1);
@@ -618,8 +620,10 @@ struct inode *ovl_get_inode(struct dentry *dentry, struct dentry *upperdentry,
* not use lower as hash key in that case.
* Hash inodes that are or could be indexed by origin inode and
* non-indexed upper inodes that could be hard linked by upper inode.
+ * Hash all inodes for NFS export.
*/
- if (!S_ISDIR(realinode->i_mode) && (upperdentry || indexed)) {
+ if (dentry->d_sb->s_export_op ||
+ (!S_ISDIR(realinode->i_mode) && (upperdentry || indexed))) {
struct inode *key = d_inode(indexed ? lowerdentry :
upperdentry);
unsigned int nlink;
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 0db59616c840..f3ad8107fec2 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -1170,6 +1170,9 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
root_dentry->d_fsdata = oe;
+ /* Hash root directory inode by upper dir inode for NFS export */
+ if (sb->s_export_op)
+ ovl_inode_update(d_inode(root_dentry), upperpath.dentry);
ovl_inode_init(d_inode(root_dentry), upperpath.dentry,
ovl_dentry_lower(root_dentry));
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 515ae156939f..c668f9e26c3e 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -276,7 +276,9 @@ void ovl_inode_update(struct inode *inode, struct dentry *upperdentry)
*/
smp_wmb();
OVL_I(inode)->__upperdentry = upperdentry;
- if (!S_ISDIR(upperinode->i_mode) && inode_unhashed(inode)) {
+ /* Hash all inodes for NFS export and non-dir for hardlink support */
+ if ((!S_ISDIR(upperinode->i_mode) || inode->i_sb->s_export_op) &&
+ inode_unhashed(inode)) {
inode->i_private = upperinode;
__insert_inode_hash(inode, (unsigned long) upperinode);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 02/14] ovl: grab i_count reference of lower inode
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
2017-10-17 16:44 ` [PATCH 01/14] ovl: hash all overlay inodes for NFS export Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 03/14] ovl: use d_splice_alias() in place of d_add() in lookup Amir Goldstein
` (13 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
An overlay inode with elevated i_count may outlive its overlay dentry.
Since the only reference to OVL_I(inode)->lower was held by the lower
dentry object in the overlay dentry (oe->lowerstack[0]), releasing the
overlay dentry could leave a live inode with reference to freed lower
inode.
This may materialize to a bug when a disconnected dentry is obtained
from decoding an NFS file handle and a new allocated real inode has
the same address as the freed inode in OVL_I(inode)->lower, by which
the overlay inode is hashed.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/super.c | 1 +
fs/overlayfs/util.c | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index f3ad8107fec2..3c98991eebcd 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -197,6 +197,7 @@ static void ovl_destroy_inode(struct inode *inode)
struct ovl_inode *oi = OVL_I(inode);
dput(oi->__upperdentry);
+ iput(oi->lower);
kfree(oi->redirect);
ovl_dir_cache_free(inode);
mutex_destroy(&oi->lock);
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index c668f9e26c3e..568ea4d4529d 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -260,7 +260,7 @@ void ovl_inode_init(struct inode *inode, struct dentry *upperdentry,
if (upperdentry)
OVL_I(inode)->__upperdentry = upperdentry;
if (lowerdentry)
- OVL_I(inode)->lower = d_inode(lowerdentry);
+ OVL_I(inode)->lower = igrab(d_inode(lowerdentry));
ovl_copyattr(d_inode(upperdentry ?: lowerdentry), inode);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 03/14] ovl: use d_splice_alias() in place of d_add() in lookup
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
2017-10-17 16:44 ` [PATCH 01/14] ovl: hash all overlay inodes for NFS export Amir Goldstein
2017-10-17 16:44 ` [PATCH 02/14] ovl: grab i_count reference of lower inode Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 04/14] ovl: copy up of disconnected dentries Amir Goldstein
` (12 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
This is required for NFS export.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/namei.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index e2b47d6494fd..0619a789809e 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -895,9 +895,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
dput(index);
kfree(stack);
kfree(d.redirect);
- d_add(dentry, inode);
-
- return NULL;
+ return d_splice_alias(inode, dentry);
out_free_oe:
dentry->d_fsdata = NULL;
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 04/14] ovl: copy up of disconnected dentries
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (2 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 03/14] ovl: use d_splice_alias() in place of d_add() in lookup Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 05/14] ovl: encode/decode pure-upper non-connectable file handles Amir Goldstein
` (11 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
With NFS export, some operations on decoded file handles (e.g. open,
link, setattr, xattr_set) may call copy up with a disconnected non-dir.
In this case, we will copy up lower inode to index dir without
linking it to upper dir.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/copy_up.c | 56 ++++++++++++++++++++++++++++++++++++--------------
fs/overlayfs/inode.c | 4 +++-
fs/overlayfs/util.c | 7 ++++---
3 files changed, 48 insertions(+), 19 deletions(-)
diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
index df1ce2d5a20e..0a2c2c3743b1 100644
--- a/fs/overlayfs/copy_up.c
+++ b/fs/overlayfs/copy_up.c
@@ -452,7 +452,10 @@ static int ovl_link_up(struct ovl_copy_up_ctx *c)
}
}
inode_unlock(udir);
- ovl_set_nlink_upper(c->dentry);
+ if (err)
+ return err;
+
+ err = ovl_set_nlink_upper(c->dentry);
return err;
}
@@ -647,6 +650,9 @@ static int ovl_do_copy_up(struct ovl_copy_up_ctx *c)
err = ovl_get_index_name(c->lowerpath.dentry, &c->destname);
if (err)
return err;
+ } else if (WARN_ON(!c->parent)) {
+ /* Disconnected dentry must be copied up to index dir */
+ return -EIO;
} else {
/*
* Mark parent "impure" because it may now contain non-pure
@@ -669,12 +675,17 @@ static int ovl_do_copy_up(struct ovl_copy_up_ctx *c)
}
}
- if (!err && c->indexed)
+
+ if (err)
+ goto out;
+
+ if (c->indexed)
ovl_set_flag(OVL_INDEX, d_inode(c->dentry));
if (to_index) {
- kfree(c->destname.name);
- } else if (!err) {
+ /* Initialize nlink for copy up of disconnected dentry */
+ err = ovl_set_nlink_upper(c->dentry);
+ } else {
struct inode *udir = d_inode(c->destdir);
/* Restore timestamps on parent (best effort) */
@@ -685,6 +696,9 @@ static int ovl_do_copy_up(struct ovl_copy_up_ctx *c)
ovl_dentry_set_upper_alias(c->dentry);
}
+out:
+ if (to_index)
+ kfree(c->destname.name);
return err;
}
@@ -709,14 +723,17 @@ static int ovl_copy_up_one(struct dentry *parent, struct dentry *dentry,
if (err)
return err;
- ovl_path_upper(parent, &parentpath);
- ctx.destdir = parentpath.dentry;
- ctx.destname = dentry->d_name;
+ if (parent) {
+ ovl_path_upper(parent, &parentpath);
+ ctx.destdir = parentpath.dentry;
+ ctx.destname = dentry->d_name;
- err = vfs_getattr(&parentpath, &ctx.pstat,
- STATX_ATIME | STATX_MTIME, AT_STATX_SYNC_AS_STAT);
- if (err)
- return err;
+ err = vfs_getattr(&parentpath, &ctx.pstat,
+ STATX_ATIME | STATX_MTIME,
+ AT_STATX_SYNC_AS_STAT);
+ if (err)
+ return err;
+ }
/* maybe truncate regular file. this has no effect on dirs */
if (flags & O_TRUNC)
@@ -737,7 +754,7 @@ static int ovl_copy_up_one(struct dentry *parent, struct dentry *dentry,
} else {
if (!ovl_dentry_upper(dentry))
err = ovl_do_copy_up(&ctx);
- if (!err && !ovl_dentry_has_upper_alias(dentry))
+ if (!err && parent && !ovl_dentry_has_upper_alias(dentry))
err = ovl_link_up(&ctx);
ovl_copy_up_end(dentry);
}
@@ -750,10 +767,19 @@ int ovl_copy_up_flags(struct dentry *dentry, int flags)
{
int err = 0;
const struct cred *old_cred = ovl_override_creds(dentry->d_sb);
+ bool disconnected = (dentry->d_flags & DCACHE_DISCONNECTED);
+
+ /*
+ * With NFS export, copy up can get called for a disconnected non-dir.
+ * In this case, we will copy up lower inode to index dir without
+ * linking it to upper dir.
+ */
+ if (WARN_ON(disconnected && d_is_dir(dentry)))
+ return -EIO;
while (!err) {
struct dentry *next;
- struct dentry *parent;
+ struct dentry *parent = NULL;
/*
* Check if copy-up has happened as well as for upper alias (in
@@ -769,12 +795,12 @@ int ovl_copy_up_flags(struct dentry *dentry, int flags)
* with rename.
*/
if (ovl_dentry_upper(dentry) &&
- ovl_dentry_has_upper_alias(dentry))
+ (ovl_dentry_has_upper_alias(dentry) || disconnected))
break;
next = dget(dentry);
/* find the topmost dentry not yet copied up */
- for (;;) {
+ for (; !disconnected;) {
parent = dget_parent(next);
if (ovl_dentry_upper(parent))
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index a792c088c2b4..5959293f8e70 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -319,8 +319,10 @@ struct posix_acl *ovl_get_acl(struct inode *inode, int type)
static bool ovl_open_need_copy_up(struct dentry *dentry, int flags)
{
+ /* Copy up of disconnected dentry does not set upper alias */
if (ovl_dentry_upper(dentry) &&
- ovl_dentry_has_upper_alias(dentry))
+ (ovl_dentry_has_upper_alias(dentry) ||
+ (dentry->d_flags & DCACHE_DISCONNECTED)))
return false;
if (special_file(d_inode(dentry)->i_mode))
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 568ea4d4529d..b71c5d1af5d1 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -209,9 +209,10 @@ void ovl_dentry_set_opaque(struct dentry *dentry)
}
/*
- * For hard links it's possible for ovl_dentry_upper() to return positive, while
- * there's no actual upper alias for the inode. Copy up code needs to know
- * about the existence of the upper alias, so it can't use ovl_dentry_upper().
+ * For hard links and decoded file handles, it's possible for ovl_dentry_upper()
+ * to return positive, while there's no actual upper alias for the inode.
+ * Copy up code needs to know about the existence of the upper alias, so it
+ * can't use ovl_dentry_upper().
*/
bool ovl_dentry_has_upper_alias(struct dentry *dentry)
{
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 05/14] ovl: encode/decode pure-upper non-connectable file handles
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (3 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 04/14] ovl: copy up of disconnected dentries Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 06/14] ovl: encode pure-upper connectable " Amir Goldstein
` (10 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Implement basic exportfs operations for pure upper files as
the ground floor of full exportfs support.
In this basic implementation, encoding an overlay file handle is
implemented by asking the underlying fs to encode the real upper inode.
Decoding is done by getting the upper dentry from underlying upper fs
and finding/creating an overlay inode in cache that is hashed by the
real upper inode.
This simplied case of exporting pure upper non-connectable file handles
is sufficient to pass xfstest generic/426 which creates files, encodes
file handles, drops caches and decodes the file handles.
Following patches will implement exportfs support for less trivial
cases.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/Makefile | 3 +-
fs/overlayfs/export.c | 140 +++++++++++++++++++++++++++++++++++++++++++++++
fs/overlayfs/inode.c | 13 ++---
fs/overlayfs/namei.c | 2 +-
fs/overlayfs/overlayfs.h | 7 ++-
5 files changed, 154 insertions(+), 11 deletions(-)
create mode 100644 fs/overlayfs/export.c
diff --git a/fs/overlayfs/Makefile b/fs/overlayfs/Makefile
index 99373bbc1478..30802347a020 100644
--- a/fs/overlayfs/Makefile
+++ b/fs/overlayfs/Makefile
@@ -4,4 +4,5 @@
obj-$(CONFIG_OVERLAY_FS) += overlay.o
-overlay-objs := super.o namei.o util.o inode.o dir.o readdir.o copy_up.o
+overlay-objs := super.o namei.o util.o inode.o dir.o readdir.o copy_up.o \
+ export.o
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
new file mode 100644
index 000000000000..33914f23530e
--- /dev/null
+++ b/fs/overlayfs/export.c
@@ -0,0 +1,140 @@
+/*
+ * Overlayfs NFS export support.
+ *
+ * Amir Goldstein <amir73il@gmail.com>
+ *
+ * Copyright (C) 2017 CTERA Networks. All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/fs.h>
+#include <linux/mount.h>
+#include <linux/xattr.h>
+#include <linux/exportfs.h>
+#include "overlayfs.h"
+#include "ovl_entry.h"
+
+/* TODO: add export_operations method dentry_to_fh() ??? */
+static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
+ int *max_len, int connectable)
+{
+ struct dentry *lower = ovl_dentry_lower(dentry);
+ int type;
+
+ /* TODO: handle encoding of non pure upper */
+ if (lower)
+ return FILEID_INVALID;
+
+ /*
+ * Ask real fs to encode the inode of the real upper dentry.
+ * When decoding we ask real fs for the upper dentry and use
+ * the real inode to get the overlay inode.
+ */
+ type = exportfs_encode_fh(ovl_dentry_upper(dentry), fid, max_len,
+ connectable);
+
+ /* TODO: encode an ovl_fh struct and return OVL file handle type */
+ return type;
+}
+
+static int ovl_encode_inode_fh(struct inode *inode, u32 *fh, int *max_len,
+ struct inode *parent)
+{
+ struct dentry *dentry = d_find_alias(inode);
+ int type;
+
+ if (!dentry)
+ return FILEID_INVALID;
+
+ /* TODO: handle encoding of non-dir connectable file handle */
+ if (parent)
+ return FILEID_INVALID;
+
+ type = ovl_dentry_to_fh(dentry, (struct fid *)fh, max_len, 0);
+
+ dput(dentry);
+ return type;
+}
+
+/*
+ * Find or instantiate an overlay dentry from real dentries.
+ * Like d_obtain_alias(inode), ovl_obtain_alias() either
+ * takes ownership on the upper dentry reference or puts it
+ * before returning an error.
+ */
+static struct dentry *ovl_obtain_alias(struct super_block *sb,
+ struct dentry *upper,
+ struct dentry *lower)
+{
+ struct inode *inode;
+ struct dentry *dentry;
+ struct ovl_entry *oe;
+
+ /* TODO: handle decoding of non pure upper */
+ if (lower) {
+ dput(upper);
+ return ERR_PTR(-EINVAL);
+ }
+
+ inode = ovl_get_inode(sb, upper, NULL, NULL);
+ if (IS_ERR(inode)) {
+ dput(upper);
+ return ERR_CAST(inode);
+ }
+
+ dentry = d_obtain_alias(inode);
+ if (IS_ERR(dentry))
+ return dentry;
+
+ if (dentry->d_fsdata) {
+ if (WARN_ON(ovl_dentry_lower(dentry) ||
+ ovl_dentry_upper(dentry)->d_inode !=
+ upper->d_inode)) {
+ dput(dentry);
+ return ERR_PTR(-ESTALE);
+ }
+ return dentry;
+ }
+
+ oe = ovl_alloc_entry(0);
+ if (!oe) {
+ dput(dentry);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ dentry->d_fsdata = oe;
+ ovl_dentry_set_upper_alias(dentry);
+
+ return dentry;
+
+}
+
+static struct dentry *ovl_fh_to_dentry(struct super_block *sb, struct fid *fid,
+ int fh_len, int fh_type)
+{
+ struct ovl_fs *ofs = sb->s_fs_info;
+ struct vfsmount *mnt = ofs->upper_mnt;
+ const struct export_operations *real_op;
+ struct dentry *upper;
+
+ /* TODO: handle decoding of non pure upper */
+ if (!mnt)
+ return NULL;
+
+ real_op = mnt->mnt_sb->s_export_op;
+ /* TODO: decode ovl_fh format file handle */
+ upper = real_op->fh_to_dentry(mnt->mnt_sb, fid, fh_len, fh_type);
+ if (IS_ERR_OR_NULL(upper))
+ return upper;
+
+ /* Find or instantiate a pure upper dentry */
+ return ovl_obtain_alias(sb, upper, NULL);
+}
+
+const struct export_operations ovl_export_operations = {
+ .encode_fh = ovl_encode_inode_fh,
+ .fh_to_dentry = ovl_fh_to_dentry,
+};
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 5959293f8e70..a623c0b60583 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -602,14 +602,13 @@ static bool ovl_verify_inode(struct inode *inode, struct dentry *lowerdentry,
return true;
}
-struct inode *ovl_get_inode(struct dentry *dentry, struct dentry *upperdentry,
- struct dentry *index)
+struct inode *ovl_get_inode(struct super_block *sb, struct dentry *upperdentry,
+ struct dentry *lowerdentry, struct dentry *index)
{
- struct dentry *lowerdentry = ovl_dentry_lower(dentry);
struct inode *realinode = upperdentry ? d_inode(upperdentry) : NULL;
struct inode *inode;
/* Already indexed or could be indexed on copy up? */
- bool indexed = (index || (ovl_indexdir(dentry->d_sb) && !upperdentry));
+ bool indexed = (index || (ovl_indexdir(sb) && !upperdentry));
if (WARN_ON(upperdentry && indexed && !lowerdentry))
return ERR_PTR(-EIO);
@@ -624,13 +623,13 @@ struct inode *ovl_get_inode(struct dentry *dentry, struct dentry *upperdentry,
* non-indexed upper inodes that could be hard linked by upper inode.
* Hash all inodes for NFS export.
*/
- if (dentry->d_sb->s_export_op ||
+ if (sb->s_export_op ||
(!S_ISDIR(realinode->i_mode) && (upperdentry || indexed))) {
struct inode *key = d_inode(indexed ? lowerdentry :
upperdentry);
unsigned int nlink;
- inode = iget5_locked(dentry->d_sb, (unsigned long) key,
+ inode = iget5_locked(sb, (unsigned long) key,
ovl_inode_test, ovl_inode_set, key);
if (!inode)
goto out_nomem;
@@ -653,7 +652,7 @@ struct inode *ovl_get_inode(struct dentry *dentry, struct dentry *upperdentry,
realinode->i_nlink);
set_nlink(inode, nlink);
} else {
- inode = new_inode(dentry->d_sb);
+ inode = new_inode(sb);
if (!inode)
goto out_nomem;
}
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 0619a789809e..ce3d4930a721 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -881,7 +881,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
upperdentry = dget(index);
if (upperdentry || ctr) {
- inode = ovl_get_inode(dentry, upperdentry, index);
+ inode = ovl_get_inode(dentry->d_sb, upperdentry, origin, index);
err = PTR_ERR(inode);
if (IS_ERR(inode))
goto out_free_oe;
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 18ea89deb040..8ad3110a9b48 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -304,8 +304,8 @@ int ovl_update_time(struct inode *inode, struct timespec *ts, int flags);
bool ovl_is_private_xattr(const char *name);
struct inode *ovl_new_inode(struct super_block *sb, umode_t mode, dev_t rdev);
-struct inode *ovl_get_inode(struct dentry *dentry, struct dentry *upperdentry,
- struct dentry *index);
+struct inode *ovl_get_inode(struct super_block *sb, struct dentry *upperdentry,
+ struct dentry *lowerdentry, struct dentry *index);
static inline void ovl_copyattr(struct inode *from, struct inode *to)
{
to->i_uid = from->i_uid;
@@ -337,3 +337,6 @@ int ovl_copy_up_flags(struct dentry *dentry, int flags);
int ovl_copy_xattr(struct dentry *old, struct dentry *new);
int ovl_set_attr(struct dentry *upper, struct kstat *stat);
struct ovl_fh *ovl_encode_fh(struct dentry *lower, bool is_upper);
+
+/* export.c */
+extern const struct export_operations ovl_export_operations;
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 06/14] ovl: encode pure-upper connectable file handles
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (4 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 05/14] ovl: encode/decode pure-upper non-connectable file handles Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-18 18:35 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 07/14] ovl: decode " Amir Goldstein
` (9 subsequent siblings)
15 siblings, 1 reply; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
To allow reconnecting of pure upper overlay dentry based on its
real parent, we restrict the implementation to encoding of overlay
dentries with pure upper ancestry up to overlay root.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/export.c | 70 ++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 61 insertions(+), 9 deletions(-)
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 33914f23530e..263415dd929b 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -17,15 +17,40 @@
#include "overlayfs.h"
#include "ovl_entry.h"
+/* Check if dentry is pure upper ancestry up to root */
+static bool ovl_is_pure_upper_or_root(struct dentry *dentry, int connectable)
+{
+ struct dentry *parent = NULL;
+
+ /* For non-connectable non-dir we don't need to check ancestry */
+ if (!d_is_dir(dentry) && !connectable)
+ return !ovl_dentry_lower(dentry);
+
+ dget(dentry);
+ while (!IS_ROOT(dentry) && !ovl_dentry_lower(dentry)) {
+ parent = dget_parent(dentry);
+ dput(dentry);
+ dentry = parent;
+ }
+ dput(dentry);
+
+ return dentry == dentry->d_sb->s_root;
+}
+
/* TODO: add export_operations method dentry_to_fh() ??? */
static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
int *max_len, int connectable)
{
- struct dentry *lower = ovl_dentry_lower(dentry);
int type;
- /* TODO: handle encoding of non pure upper */
- if (lower)
+ /*
+ * Overlay root dir inode is hashed and encoded as pure upper, because
+ * root dir dentry is born upper and not indexed. It is not a problem
+ * that root dir is not indexed, because root dentry is pinned to cache.
+ *
+ * TODO: handle encoding of non pure upper.
+ */
+ if (!ovl_is_pure_upper_or_root(dentry, connectable))
return FILEID_INVALID;
/*
@@ -40,20 +65,47 @@ static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
return type;
}
+/* Find an alias of inode. If @dir is non NULL, find a child alias */
+static struct dentry *ovl_find_alias(struct inode *inode, struct inode *dir)
+{
+ struct dentry *parent, *child;
+ struct dentry *alias = NULL;
+
+ /* Parent inode is never provided when encoding a directory */
+ if (!dir || WARN_ON(!S_ISDIR(dir->i_mode) || S_ISDIR(inode->i_mode)))
+ return d_find_alias(inode);
+
+ /*
+ * Run all of the dentries associated with this parent. Since this is
+ * a directory, there damn well better only be one item on this list.
+ */
+ spin_lock(&dir->i_lock);
+ hlist_for_each_entry(parent, &dir->i_dentry, d_u.d_alias) {
+ /* Find an alias of inode who is a child of parent */
+ spin_lock(&parent->d_lock);
+ list_for_each_entry(child, &parent->d_subdirs, d_child) {
+ if (child->d_inode == inode) {
+ alias = dget(child);
+ break;
+ }
+ }
+ spin_unlock(&parent->d_lock);
+ }
+ spin_unlock(&dir->i_lock);
+
+ return alias;
+}
+
static int ovl_encode_inode_fh(struct inode *inode, u32 *fh, int *max_len,
struct inode *parent)
{
- struct dentry *dentry = d_find_alias(inode);
+ struct dentry *dentry = ovl_find_alias(inode, parent);
int type;
if (!dentry)
return FILEID_INVALID;
- /* TODO: handle encoding of non-dir connectable file handle */
- if (parent)
- return FILEID_INVALID;
-
- type = ovl_dentry_to_fh(dentry, (struct fid *)fh, max_len, 0);
+ type = ovl_dentry_to_fh(dentry, (struct fid *)fh, max_len, !!parent);
dput(dentry);
return type;
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH 06/14] ovl: encode pure-upper connectable file handles
2017-10-17 16:44 ` [PATCH 06/14] ovl: encode pure-upper connectable " Amir Goldstein
@ 2017-10-18 18:35 ` Amir Goldstein
0 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-18 18:35 UTC (permalink / raw)
To: Miklos Szeredi; +Cc: Jeff Layton, J . Bruce Fields, overlayfs, linux-fsdevel
On Tue, Oct 17, 2017 at 7:44 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> To allow reconnecting of pure upper overlay dentry based on its
> real parent, we restrict the implementation to encoding of overlay
> dentries with pure upper ancestry up to overlay root.
>
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> ---
> fs/overlayfs/export.c | 70 ++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 61 insertions(+), 9 deletions(-)
>
> diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
> index 33914f23530e..263415dd929b 100644
> --- a/fs/overlayfs/export.c
> +++ b/fs/overlayfs/export.c
> @@ -17,15 +17,40 @@
> #include "overlayfs.h"
> #include "ovl_entry.h"
>
> +/* Check if dentry is pure upper ancestry up to root */
> +static bool ovl_is_pure_upper_or_root(struct dentry *dentry, int connectable)
> +{
> + struct dentry *parent = NULL;
> +
> + /* For non-connectable non-dir we don't need to check ancestry */
> + if (!d_is_dir(dentry) && !connectable)
> + return !ovl_dentry_lower(dentry);
> +
> + dget(dentry);
> + while (!IS_ROOT(dentry) && !ovl_dentry_lower(dentry)) {
> + parent = dget_parent(dentry);
> + dput(dentry);
> + dentry = parent;
> + }
> + dput(dentry);
> +
> + return dentry == dentry->d_sb->s_root;
> +}
> +
> /* TODO: add export_operations method dentry_to_fh() ??? */
> static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
> int *max_len, int connectable)
> {
> - struct dentry *lower = ovl_dentry_lower(dentry);
> int type;
>
> - /* TODO: handle encoding of non pure upper */
> - if (lower)
> + /*
> + * Overlay root dir inode is hashed and encoded as pure upper, because
> + * root dir dentry is born upper and not indexed. It is not a problem
> + * that root dir is not indexed, because root dentry is pinned to cache.
> + *
> + * TODO: handle encoding of non pure upper.
> + */
> + if (!ovl_is_pure_upper_or_root(dentry, connectable))
> return FILEID_INVALID;
>
> /*
> @@ -40,20 +65,47 @@ static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
> return type;
> }
>
> +/* Find an alias of inode. If @dir is non NULL, find a child alias */
> +static struct dentry *ovl_find_alias(struct inode *inode, struct inode *dir)
> +{
> + struct dentry *parent, *child;
> + struct dentry *alias = NULL;
> +
> + /* Parent inode is never provided when encoding a directory */
> + if (!dir || WARN_ON(!S_ISDIR(dir->i_mode) || S_ISDIR(inode->i_mode)))
> + return d_find_alias(inode);
> +
> + /*
> + * Run all of the dentries associated with this parent. Since this is
> + * a directory, there damn well better only be one item on this list.
> + */
FYI, this iteration is breaking locking order.
I pushed a fix patch to iterate on aliases looking for match with parent
to branch ovl-nfs-export-v1.
This code was not exercised by my xfstests because name_to_handle_at()
does not encode connectable file handles.
I use a test patch (at the tip of ovl-nfs-export-v1) to encode connectable
file handles from name_to_handle_at() to test this code.
> + spin_lock(&dir->i_lock);
> + hlist_for_each_entry(parent, &dir->i_dentry, d_u.d_alias) {
> + /* Find an alias of inode who is a child of parent */
> + spin_lock(&parent->d_lock);
> + list_for_each_entry(child, &parent->d_subdirs, d_child) {
> + if (child->d_inode == inode) {
> + alias = dget(child);
> + break;
> + }
> + }
> + spin_unlock(&parent->d_lock);
> + }
> + spin_unlock(&dir->i_lock);
> +
> + return alias;
> +}
> +
> static int ovl_encode_inode_fh(struct inode *inode, u32 *fh, int *max_len,
> struct inode *parent)
> {
> - struct dentry *dentry = d_find_alias(inode);
> + struct dentry *dentry = ovl_find_alias(inode, parent);
> int type;
>
> if (!dentry)
> return FILEID_INVALID;
>
> - /* TODO: handle encoding of non-dir connectable file handle */
> - if (parent)
> - return FILEID_INVALID;
> -
> - type = ovl_dentry_to_fh(dentry, (struct fid *)fh, max_len, 0);
> + type = ovl_dentry_to_fh(dentry, (struct fid *)fh, max_len, !!parent);
>
> dput(dentry);
> return type;
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 07/14] ovl: decode pure-upper connectable file handles
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (5 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 06/14] ovl: encode pure-upper connectable " Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 08/14] ovl: encode/decode struct ovl_fh format " Amir Goldstein
` (8 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Implement fh_to_parent() and get_parent() operations to reconnect
a pure upper overlay dentry.
This fixes a variant of xfstest test generic/426 which creates a
directory, encodes its file handle, drops caches and decodes it.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/export.c | 53 +++++++++++++++++++++++++++++++++++++++++++-----
fs/overlayfs/namei.c | 5 -----
fs/overlayfs/overlayfs.h | 5 +++++
3 files changed, 53 insertions(+), 10 deletions(-)
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 263415dd929b..9ec2fe659e38 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -138,7 +138,7 @@ static struct dentry *ovl_obtain_alias(struct super_block *sb,
}
dentry = d_obtain_alias(inode);
- if (IS_ERR(dentry))
+ if (IS_ERR(dentry) || dentry == dentry->d_sb->s_root)
return dentry;
if (dentry->d_fsdata) {
@@ -159,26 +159,33 @@ static struct dentry *ovl_obtain_alias(struct super_block *sb,
dentry->d_fsdata = oe;
ovl_dentry_set_upper_alias(dentry);
+ if (d_is_dir(upper) && ovl_is_opaquedir(upper))
+ ovl_dentry_set_opaque(dentry);
return dentry;
}
-static struct dentry *ovl_fh_to_dentry(struct super_block *sb, struct fid *fid,
- int fh_len, int fh_type)
+static struct dentry *ovl_fh_to_d(struct super_block *sb, struct fid *fid,
+ int fh_len, int fh_type, bool to_parent)
{
struct ovl_fs *ofs = sb->s_fs_info;
struct vfsmount *mnt = ofs->upper_mnt;
const struct export_operations *real_op;
+ struct dentry *(*fh_to_d)(struct super_block *, struct fid *, int, int);
struct dentry *upper;
/* TODO: handle decoding of non pure upper */
- if (!mnt)
+ if (!mnt || !mnt->mnt_sb->s_export_op)
return NULL;
real_op = mnt->mnt_sb->s_export_op;
+ fh_to_d = to_parent ? real_op->fh_to_parent : real_op->fh_to_dentry;
+ if (!fh_to_d)
+ return NULL;
+
/* TODO: decode ovl_fh format file handle */
- upper = real_op->fh_to_dentry(mnt->mnt_sb, fid, fh_len, fh_type);
+ upper = fh_to_d(mnt->mnt_sb, fid, fh_len, fh_type);
if (IS_ERR_OR_NULL(upper))
return upper;
@@ -186,7 +193,43 @@ static struct dentry *ovl_fh_to_dentry(struct super_block *sb, struct fid *fid,
return ovl_obtain_alias(sb, upper, NULL);
}
+static struct dentry *ovl_fh_to_dentry(struct super_block *sb, struct fid *fid,
+ int fh_len, int fh_type)
+{
+ return ovl_fh_to_d(sb, fid, fh_len, fh_type, false);
+}
+
+static struct dentry *ovl_fh_to_parent(struct super_block *sb, struct fid *fid,
+ int fh_len, int fh_type)
+{
+ return ovl_fh_to_d(sb, fid, fh_len, fh_type, true);
+}
+
+static struct dentry *ovl_get_parent(struct dentry *dentry)
+{
+ const struct export_operations *real_op;
+ struct dentry *upper;
+
+ /* TODO: handle connecting of non pure upper */
+ if (ovl_dentry_lower(dentry))
+ return ERR_PTR(-EACCES);
+
+ upper = ovl_dentry_upper(dentry);
+ real_op = upper->d_sb->s_export_op;
+ if (!real_op || !real_op->get_parent)
+ return ERR_PTR(-EACCES);
+
+ upper = real_op->get_parent(upper);
+ if (IS_ERR(upper))
+ return upper;
+
+ /* Find or instantiate a pure upper dentry */
+ return ovl_obtain_alias(dentry->d_sb, upper, NULL);
+}
+
const struct export_operations ovl_export_operations = {
.encode_fh = ovl_encode_inode_fh,
.fh_to_dentry = ovl_fh_to_dentry,
+ .fh_to_parent = ovl_fh_to_parent,
+ .get_parent = ovl_get_parent,
};
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index ce3d4930a721..d5313fb02e73 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -209,11 +209,6 @@ static struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt)
return origin;
}
-static bool ovl_is_opaquedir(struct dentry *dentry)
-{
- return ovl_check_dir_xattr(dentry, OVL_XATTR_OPAQUE);
-}
-
static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
const char *name, unsigned int namelen,
size_t prelen, const char *post,
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 8ad3110a9b48..66a6447a0c2a 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -259,6 +259,11 @@ static inline bool ovl_is_impuredir(struct dentry *dentry)
return ovl_check_dir_xattr(dentry, OVL_XATTR_IMPURE);
}
+static inline bool ovl_is_opaquedir(struct dentry *dentry)
+{
+ return ovl_check_dir_xattr(dentry, OVL_XATTR_OPAQUE);
+}
+
/* namei.c */
int ovl_verify_origin(struct dentry *dentry, struct dentry *origin,
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 08/14] ovl: encode/decode struct ovl_fh format file handles
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (6 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 07/14] ovl: decode " Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-18 18:31 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 09/14] ovl: encode non-pure-upper non-connectable " Amir Goldstein
` (7 subsequent siblings)
15 siblings, 1 reply; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Encode an overlay inode as struct ovl_fh encoding of the real inode.
Pure upper file handles are explicitly marked with a flag in ovl_fh
header.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/copy_up.c | 18 +++++-----
fs/overlayfs/export.c | 86 +++++++++++++++++++++++++++++++++---------------
fs/overlayfs/namei.c | 12 ++++---
fs/overlayfs/overlayfs.h | 10 +++++-
4 files changed, 85 insertions(+), 41 deletions(-)
diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
index 0a2c2c3743b1..9af3a6b62038 100644
--- a/fs/overlayfs/copy_up.c
+++ b/fs/overlayfs/copy_up.c
@@ -233,25 +233,21 @@ int ovl_set_attr(struct dentry *upperdentry, struct kstat *stat)
return err;
}
-struct ovl_fh *ovl_encode_fh(struct dentry *lower, bool is_upper)
+struct ovl_fh *ovl_encode_fh(struct dentry *dentry, bool is_upper,
+ bool connectable)
{
struct ovl_fh *fh;
int fh_type, fh_len, dwords;
void *buf;
int buflen = MAX_HANDLE_SZ;
- uuid_t *uuid = &lower->d_sb->s_uuid;
+ uuid_t *uuid = &dentry->d_sb->s_uuid;
buf = kmalloc(buflen, GFP_KERNEL);
if (!buf)
return ERR_PTR(-ENOMEM);
- /*
- * We encode a non-connectable file handle for non-dir, because we
- * only need to find the lower inode number and we don't want to pay
- * the price or reconnecting the dentry.
- */
dwords = buflen >> 2;
- fh_type = exportfs_encode_fh(lower, buf, &dwords, 0);
+ fh_type = exportfs_encode_fh(dentry, buf, &dwords, connectable);
buflen = (dwords << 2);
fh = ERR_PTR(-EIO);
@@ -296,12 +292,16 @@ static int ovl_set_origin(struct dentry *dentry, struct dentry *origin,
int err;
/*
+ * We encode a non-connectable file handle for non-dir, because we
+ * only need to find the lower inode number and we don't want to pay
+ * the price of reconnecting the dentry on lookup.
+ *
* When lower layer doesn't support export operations store a 'null' fh,
* so we can use the overlay.origin xattr to distignuish between a copy
* up and a pure upper inode.
*/
if (ovl_can_decode_fh(origin->d_sb)) {
- fh = ovl_encode_fh(origin, is_upper);
+ fh = ovl_encode_fh(origin, is_upper, false);
if (IS_ERR(fh))
return PTR_ERR(fh);
}
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 9ec2fe659e38..47ef0302dbd3 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -41,7 +41,8 @@ static bool ovl_is_pure_upper_or_root(struct dentry *dentry, int connectable)
static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
int *max_len, int connectable)
{
- int type;
+ const struct ovl_fh *fh;
+ int len = *max_len << 2;
/*
* Overlay root dir inode is hashed and encoded as pure upper, because
@@ -49,20 +50,27 @@ static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
* that root dir is not indexed, because root dentry is pinned to cache.
*
* TODO: handle encoding of non pure upper.
+ * Parent and child may not be on the same layer, so encode
+ * connectable file handle as an array of self ovl_fh and
+ * parent ovl_fh (type OVL_FILEID_WITH_PARENT).
*/
if (!ovl_is_pure_upper_or_root(dentry, connectable))
return FILEID_INVALID;
- /*
- * Ask real fs to encode the inode of the real upper dentry.
- * When decoding we ask real fs for the upper dentry and use
- * the real inode to get the overlay inode.
- */
- type = exportfs_encode_fh(ovl_dentry_upper(dentry), fid, max_len,
- connectable);
+ fh = ovl_encode_fh(ovl_dentry_upper(dentry), true, connectable);
+ if (IS_ERR(fh))
+ return FILEID_INVALID;
- /* TODO: encode an ovl_fh struct and return OVL file handle type */
- return type;
+ if (fh->len > len) {
+ kfree(fh);
+ return FILEID_INVALID;
+ }
+
+ memcpy((char *)fid, (char *)fh, len);
+ *max_len = len >> 2;
+ kfree(fh);
+
+ return OVL_FILEID_WITHOUT_PARENT;
}
/* Find an alias of inode. If @dir is non NULL, find a child alias */
@@ -171,24 +179,44 @@ static struct dentry *ovl_fh_to_d(struct super_block *sb, struct fid *fid,
{
struct ovl_fs *ofs = sb->s_fs_info;
struct vfsmount *mnt = ofs->upper_mnt;
- const struct export_operations *real_op;
- struct dentry *(*fh_to_d)(struct super_block *, struct fid *, int, int);
struct dentry *upper;
+ struct ovl_fh *fh = (struct ovl_fh *) fid;
+ int err;
- /* TODO: handle decoding of non pure upper */
- if (!mnt || !mnt->mnt_sb->s_export_op)
- return NULL;
+ /* TODO: handle file handle with parent from different layer */
+ if (fh_type != OVL_FILEID_WITHOUT_PARENT)
+ return ERR_PTR(-EINVAL);
+
+ err = ovl_check_fh_len(fh, fh_len << 2);
+ if (err)
+ return ERR_PTR(err);
- real_op = mnt->mnt_sb->s_export_op;
- fh_to_d = to_parent ? real_op->fh_to_parent : real_op->fh_to_dentry;
- if (!fh_to_d)
+ /* TODO: handle decoding of non pure upper */
+ if (!mnt || !(fh->flags & OVL_FH_FLAG_PATH_UPPER))
return NULL;
- /* TODO: decode ovl_fh format file handle */
- upper = fh_to_d(mnt->mnt_sb, fid, fh_len, fh_type);
+ upper = ovl_decode_fh(fh, mnt);
if (IS_ERR_OR_NULL(upper))
return upper;
+ /*
+ * ovl_decode_fh() will return a connected dentry if the encoded real
+ * file handle was connectable (the case of pure upper ancestry).
+ * fh_to_parent() needs to instantiate an overlay dentry from real
+ * upper parent in that case.
+ */
+ if (to_parent) {
+ struct dentry *parent;
+
+ if (upper->d_flags & DCACHE_DISCONNECTED) {
+ dput(upper);
+ return NULL;
+ }
+ parent = dget_parent(upper);
+ dput(upper);
+ upper = parent;
+ }
+
/* Find or instantiate a pure upper dentry */
return ovl_obtain_alias(sb, upper, NULL);
}
@@ -207,21 +235,25 @@ static struct dentry *ovl_fh_to_parent(struct super_block *sb, struct fid *fid,
static struct dentry *ovl_get_parent(struct dentry *dentry)
{
- const struct export_operations *real_op;
struct dentry *upper;
/* TODO: handle connecting of non pure upper */
if (ovl_dentry_lower(dentry))
return ERR_PTR(-EACCES);
+ /*
+ * When ovl_fh_to_d() returns an overlay dentry, its real upper
+ * dentry should be positive and connected. The reconnecting of
+ * the upper dentry is done by ovl_decode_fh() when decoding the
+ * real upper file handle, so here we have the upper dentry parent
+ * and we need to instantiate an overlay dentry with upper dentry
+ * parent.
+ */
upper = ovl_dentry_upper(dentry);
- real_op = upper->d_sb->s_export_op;
- if (!real_op || !real_op->get_parent)
- return ERR_PTR(-EACCES);
+ if (!upper || (upper->d_flags & DCACHE_DISCONNECTED))
+ return ERR_PTR(-ESTALE);
- upper = real_op->get_parent(upper);
- if (IS_ERR(upper))
- return upper;
+ upper = dget_parent(upper);
/* Find or instantiate a pure upper dentry */
return ovl_obtain_alias(dentry->d_sb, upper, NULL);
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index d5313fb02e73..4f20af22bd0b 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -108,7 +108,7 @@ static int ovl_acceptable(void *mnt, struct dentry *dentry)
* Return -ENODATA for "origin unknown".
* Return <0 for an invalid file handle.
*/
-static int ovl_check_fh_len(struct ovl_fh *fh, int fh_len)
+int ovl_check_fh_len(struct ovl_fh *fh, int fh_len)
{
if (fh_len < sizeof(struct ovl_fh) || fh_len < fh->len)
return -EINVAL;
@@ -172,7 +172,7 @@ static struct ovl_fh *ovl_get_origin_fh(struct dentry *dentry)
goto out;
}
-static struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt)
+struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt)
{
struct dentry *origin;
int bytes;
@@ -412,7 +412,7 @@ int ovl_verify_origin(struct dentry *dentry, struct dentry *origin,
struct ovl_fh *fh;
int err;
- fh = ovl_encode_fh(origin, is_upper);
+ fh = ovl_encode_fh(origin, is_upper, false);
err = PTR_ERR(fh);
if (IS_ERR(fh))
goto fail;
@@ -574,7 +574,11 @@ int ovl_get_index_name(struct dentry *origin, struct qstr *name)
struct ovl_fh *fh;
char *n, *s;
- fh = ovl_encode_fh(origin, false);
+ /*
+ * We encode a non-connectable file handle for index, because the index
+ * must be unqiue and invariant of lower hardlink aliases.
+ */
+ fh = ovl_encode_fh(origin, false, false);
if (IS_ERR(fh))
return PTR_ERR(fh);
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 66a6447a0c2a..75a8b10d4e10 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -72,6 +72,11 @@ enum ovl_index {
#error Endianness not defined
#endif
+/* The type returned by overlay encode_fh when encoding an ovl_fh handle */
+#define OVL_FILEID_WITHOUT_PARENT 0xf1
+/* The type returned by overlay encode_fh when encoding two ovl_fh handles */
+#define OVL_FILEID_WITH_PARENT 0xf2
+
/* On-disk and in-memeory format for redirect by file handle */
struct ovl_fh {
u8 version; /* 0 */
@@ -266,6 +271,8 @@ static inline bool ovl_is_opaquedir(struct dentry *dentry)
/* namei.c */
+int ovl_check_fh_len(struct ovl_fh *fh, int fh_len);
+struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt);
int ovl_verify_origin(struct dentry *dentry, struct dentry *origin,
bool is_upper, bool set);
int ovl_verify_index(struct dentry *index, struct vfsmount *mnt,
@@ -341,7 +348,8 @@ int ovl_copy_up(struct dentry *dentry);
int ovl_copy_up_flags(struct dentry *dentry, int flags);
int ovl_copy_xattr(struct dentry *old, struct dentry *new);
int ovl_set_attr(struct dentry *upper, struct kstat *stat);
-struct ovl_fh *ovl_encode_fh(struct dentry *lower, bool is_upper);
+struct ovl_fh *ovl_encode_fh(struct dentry *dentry, bool is_upper,
+ bool connectable);
/* export.c */
extern const struct export_operations ovl_export_operations;
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH 08/14] ovl: encode/decode struct ovl_fh format file handles
2017-10-17 16:44 ` [PATCH 08/14] ovl: encode/decode struct ovl_fh format " Amir Goldstein
@ 2017-10-18 18:31 ` Amir Goldstein
0 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-18 18:31 UTC (permalink / raw)
To: Miklos Szeredi; +Cc: Jeff Layton, J . Bruce Fields, overlayfs, linux-fsdevel
On Tue, Oct 17, 2017 at 7:44 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> Encode an overlay inode as struct ovl_fh encoding of the real inode.
> Pure upper file handles are explicitly marked with a flag in ovl_fh
> header.
>
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> ---
> fs/overlayfs/copy_up.c | 18 +++++-----
> fs/overlayfs/export.c | 86 +++++++++++++++++++++++++++++++++---------------
> fs/overlayfs/namei.c | 12 ++++---
> fs/overlayfs/overlayfs.h | 10 +++++-
> 4 files changed, 85 insertions(+), 41 deletions(-)
>
> diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
> index 0a2c2c3743b1..9af3a6b62038 100644
> --- a/fs/overlayfs/copy_up.c
> +++ b/fs/overlayfs/copy_up.c
> @@ -233,25 +233,21 @@ int ovl_set_attr(struct dentry *upperdentry, struct kstat *stat)
> return err;
> }
>
> -struct ovl_fh *ovl_encode_fh(struct dentry *lower, bool is_upper)
> +struct ovl_fh *ovl_encode_fh(struct dentry *dentry, bool is_upper,
> + bool connectable)
> {
> struct ovl_fh *fh;
> int fh_type, fh_len, dwords;
> void *buf;
> int buflen = MAX_HANDLE_SZ;
> - uuid_t *uuid = &lower->d_sb->s_uuid;
> + uuid_t *uuid = &dentry->d_sb->s_uuid;
>
> buf = kmalloc(buflen, GFP_KERNEL);
> if (!buf)
> return ERR_PTR(-ENOMEM);
>
> - /*
> - * We encode a non-connectable file handle for non-dir, because we
> - * only need to find the lower inode number and we don't want to pay
> - * the price or reconnecting the dentry.
> - */
> dwords = buflen >> 2;
> - fh_type = exportfs_encode_fh(lower, buf, &dwords, 0);
> + fh_type = exportfs_encode_fh(dentry, buf, &dwords, connectable);
> buflen = (dwords << 2);
>
> fh = ERR_PTR(-EIO);
> @@ -296,12 +292,16 @@ static int ovl_set_origin(struct dentry *dentry, struct dentry *origin,
> int err;
>
> /*
> + * We encode a non-connectable file handle for non-dir, because we
> + * only need to find the lower inode number and we don't want to pay
> + * the price of reconnecting the dentry on lookup.
> + *
> * When lower layer doesn't support export operations store a 'null' fh,
> * so we can use the overlay.origin xattr to distignuish between a copy
> * up and a pure upper inode.
> */
> if (ovl_can_decode_fh(origin->d_sb)) {
> - fh = ovl_encode_fh(origin, is_upper);
> + fh = ovl_encode_fh(origin, is_upper, false);
> if (IS_ERR(fh))
> return PTR_ERR(fh);
> }
> diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
> index 9ec2fe659e38..47ef0302dbd3 100644
> --- a/fs/overlayfs/export.c
> +++ b/fs/overlayfs/export.c
> @@ -41,7 +41,8 @@ static bool ovl_is_pure_upper_or_root(struct dentry *dentry, int connectable)
> static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
> int *max_len, int connectable)
> {
> - int type;
> + const struct ovl_fh *fh;
> + int len = *max_len << 2;
>
> /*
> * Overlay root dir inode is hashed and encoded as pure upper, because
> @@ -49,20 +50,27 @@ static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
> * that root dir is not indexed, because root dentry is pinned to cache.
> *
> * TODO: handle encoding of non pure upper.
> + * Parent and child may not be on the same layer, so encode
> + * connectable file handle as an array of self ovl_fh and
> + * parent ovl_fh (type OVL_FILEID_WITH_PARENT).
> */
> if (!ovl_is_pure_upper_or_root(dentry, connectable))
> return FILEID_INVALID;
>
> - /*
> - * Ask real fs to encode the inode of the real upper dentry.
> - * When decoding we ask real fs for the upper dentry and use
> - * the real inode to get the overlay inode.
> - */
> - type = exportfs_encode_fh(ovl_dentry_upper(dentry), fid, max_len,
> - connectable);
> + fh = ovl_encode_fh(ovl_dentry_upper(dentry), true, connectable);
> + if (IS_ERR(fh))
> + return FILEID_INVALID;
>
> - /* TODO: encode an ovl_fh struct and return OVL file handle type */
> - return type;
> + if (fh->len > len) {
> + kfree(fh);
> + return FILEID_INVALID;
> + }
> +
> + memcpy((char *)fid, (char *)fh, len);
> + *max_len = len >> 2;
FYI, there are 2 bugs cancelling each other here.
memcpy should copy fh->len
and max_len should be round up to dwords from fh->len
I pushed a fix patch to branch ovl-nfs-export-v1
> + kfree(fh);
> +
> + return OVL_FILEID_WITHOUT_PARENT;
> }
>
> /* Find an alias of inode. If @dir is non NULL, find a child alias */
> @@ -171,24 +179,44 @@ static struct dentry *ovl_fh_to_d(struct super_block *sb, struct fid *fid,
> {
> struct ovl_fs *ofs = sb->s_fs_info;
> struct vfsmount *mnt = ofs->upper_mnt;
> - const struct export_operations *real_op;
> - struct dentry *(*fh_to_d)(struct super_block *, struct fid *, int, int);
> struct dentry *upper;
> + struct ovl_fh *fh = (struct ovl_fh *) fid;
> + int err;
>
> - /* TODO: handle decoding of non pure upper */
> - if (!mnt || !mnt->mnt_sb->s_export_op)
> - return NULL;
> + /* TODO: handle file handle with parent from different layer */
> + if (fh_type != OVL_FILEID_WITHOUT_PARENT)
> + return ERR_PTR(-EINVAL);
> +
> + err = ovl_check_fh_len(fh, fh_len << 2);
> + if (err)
> + return ERR_PTR(err);
>
> - real_op = mnt->mnt_sb->s_export_op;
> - fh_to_d = to_parent ? real_op->fh_to_parent : real_op->fh_to_dentry;
> - if (!fh_to_d)
> + /* TODO: handle decoding of non pure upper */
> + if (!mnt || !(fh->flags & OVL_FH_FLAG_PATH_UPPER))
> return NULL;
>
> - /* TODO: decode ovl_fh format file handle */
> - upper = fh_to_d(mnt->mnt_sb, fid, fh_len, fh_type);
> + upper = ovl_decode_fh(fh, mnt);
> if (IS_ERR_OR_NULL(upper))
> return upper;
>
> + /*
> + * ovl_decode_fh() will return a connected dentry if the encoded real
> + * file handle was connectable (the case of pure upper ancestry).
> + * fh_to_parent() needs to instantiate an overlay dentry from real
> + * upper parent in that case.
> + */
> + if (to_parent) {
> + struct dentry *parent;
> +
> + if (upper->d_flags & DCACHE_DISCONNECTED) {
> + dput(upper);
> + return NULL;
> + }
> + parent = dget_parent(upper);
> + dput(upper);
> + upper = parent;
> + }
> +
> /* Find or instantiate a pure upper dentry */
> return ovl_obtain_alias(sb, upper, NULL);
> }
> @@ -207,21 +235,25 @@ static struct dentry *ovl_fh_to_parent(struct super_block *sb, struct fid *fid,
>
> static struct dentry *ovl_get_parent(struct dentry *dentry)
> {
> - const struct export_operations *real_op;
> struct dentry *upper;
>
> /* TODO: handle connecting of non pure upper */
> if (ovl_dentry_lower(dentry))
> return ERR_PTR(-EACCES);
>
> + /*
> + * When ovl_fh_to_d() returns an overlay dentry, its real upper
> + * dentry should be positive and connected. The reconnecting of
> + * the upper dentry is done by ovl_decode_fh() when decoding the
> + * real upper file handle, so here we have the upper dentry parent
> + * and we need to instantiate an overlay dentry with upper dentry
> + * parent.
> + */
> upper = ovl_dentry_upper(dentry);
> - real_op = upper->d_sb->s_export_op;
> - if (!real_op || !real_op->get_parent)
> - return ERR_PTR(-EACCES);
> + if (!upper || (upper->d_flags & DCACHE_DISCONNECTED))
> + return ERR_PTR(-ESTALE);
>
> - upper = real_op->get_parent(upper);
> - if (IS_ERR(upper))
> - return upper;
> + upper = dget_parent(upper);
>
> /* Find or instantiate a pure upper dentry */
> return ovl_obtain_alias(dentry->d_sb, upper, NULL);
> diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
> index d5313fb02e73..4f20af22bd0b 100644
> --- a/fs/overlayfs/namei.c
> +++ b/fs/overlayfs/namei.c
> @@ -108,7 +108,7 @@ static int ovl_acceptable(void *mnt, struct dentry *dentry)
> * Return -ENODATA for "origin unknown".
> * Return <0 for an invalid file handle.
> */
> -static int ovl_check_fh_len(struct ovl_fh *fh, int fh_len)
> +int ovl_check_fh_len(struct ovl_fh *fh, int fh_len)
> {
> if (fh_len < sizeof(struct ovl_fh) || fh_len < fh->len)
> return -EINVAL;
> @@ -172,7 +172,7 @@ static struct ovl_fh *ovl_get_origin_fh(struct dentry *dentry)
> goto out;
> }
>
> -static struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt)
> +struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt)
> {
> struct dentry *origin;
> int bytes;
> @@ -412,7 +412,7 @@ int ovl_verify_origin(struct dentry *dentry, struct dentry *origin,
> struct ovl_fh *fh;
> int err;
>
> - fh = ovl_encode_fh(origin, is_upper);
> + fh = ovl_encode_fh(origin, is_upper, false);
> err = PTR_ERR(fh);
> if (IS_ERR(fh))
> goto fail;
> @@ -574,7 +574,11 @@ int ovl_get_index_name(struct dentry *origin, struct qstr *name)
> struct ovl_fh *fh;
> char *n, *s;
>
> - fh = ovl_encode_fh(origin, false);
> + /*
> + * We encode a non-connectable file handle for index, because the index
> + * must be unqiue and invariant of lower hardlink aliases.
> + */
> + fh = ovl_encode_fh(origin, false, false);
> if (IS_ERR(fh))
> return PTR_ERR(fh);
>
> diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
> index 66a6447a0c2a..75a8b10d4e10 100644
> --- a/fs/overlayfs/overlayfs.h
> +++ b/fs/overlayfs/overlayfs.h
> @@ -72,6 +72,11 @@ enum ovl_index {
> #error Endianness not defined
> #endif
>
> +/* The type returned by overlay encode_fh when encoding an ovl_fh handle */
> +#define OVL_FILEID_WITHOUT_PARENT 0xf1
> +/* The type returned by overlay encode_fh when encoding two ovl_fh handles */
> +#define OVL_FILEID_WITH_PARENT 0xf2
> +
> /* On-disk and in-memeory format for redirect by file handle */
> struct ovl_fh {
> u8 version; /* 0 */
> @@ -266,6 +271,8 @@ static inline bool ovl_is_opaquedir(struct dentry *dentry)
>
>
> /* namei.c */
> +int ovl_check_fh_len(struct ovl_fh *fh, int fh_len);
> +struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt);
> int ovl_verify_origin(struct dentry *dentry, struct dentry *origin,
> bool is_upper, bool set);
> int ovl_verify_index(struct dentry *index, struct vfsmount *mnt,
> @@ -341,7 +348,8 @@ int ovl_copy_up(struct dentry *dentry);
> int ovl_copy_up_flags(struct dentry *dentry, int flags);
> int ovl_copy_xattr(struct dentry *old, struct dentry *new);
> int ovl_set_attr(struct dentry *upper, struct kstat *stat);
> -struct ovl_fh *ovl_encode_fh(struct dentry *lower, bool is_upper);
> +struct ovl_fh *ovl_encode_fh(struct dentry *dentry, bool is_upper,
> + bool connectable);
>
> /* export.c */
> extern const struct export_operations ovl_export_operations;
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 09/14] ovl: encode non-pure-upper non-connectable file handles
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (7 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 08/14] ovl: encode/decode struct ovl_fh format " Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 10/14] ovl: obtain a non-pure-upper disconnected dentry Amir Goldstein
` (6 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Encode a non pure upper overlay inode as struct ovl_fh encoding
of the copy up origin inode. The encoded real inode, is also the
inode used as the key to hash the overlay inode.
We can only encode upper with origin if it is indexed, so NFS export
will work only if overlay was mounted with index=all from the start.
Copy up directory on encode to create an index. We need the index
to decode a connected upper dir dentry, which we will use to
reconnect a disconnected overlay dir dentry.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/export.c | 65 ++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 56 insertions(+), 9 deletions(-)
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 47ef0302dbd3..7ebe04317647 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -18,14 +18,10 @@
#include "ovl_entry.h"
/* Check if dentry is pure upper ancestry up to root */
-static bool ovl_is_pure_upper_or_root(struct dentry *dentry, int connectable)
+static bool ovl_is_pure_upper_or_root(struct dentry *dentry)
{
struct dentry *parent = NULL;
- /* For non-connectable non-dir we don't need to check ancestry */
- if (!d_is_dir(dentry) && !connectable)
- return !ovl_dentry_lower(dentry);
-
dget(dentry);
while (!IS_ROOT(dentry) && !ovl_dentry_lower(dentry)) {
parent = dget_parent(dentry);
@@ -41,6 +37,8 @@ static bool ovl_is_pure_upper_or_root(struct dentry *dentry, int connectable)
static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
int *max_len, int connectable)
{
+ struct dentry *origin = ovl_dentry_lower(dentry);
+ struct dentry *upper = ovl_dentry_upper(dentry);
const struct ovl_fh *fh;
int len = *max_len << 2;
@@ -48,16 +46,65 @@ static int ovl_dentry_to_fh(struct dentry *dentry, struct fid *fid,
* Overlay root dir inode is hashed and encoded as pure upper, because
* root dir dentry is born upper and not indexed. It is not a problem
* that root dir is not indexed, because root dentry is pinned to cache.
- *
- * TODO: handle encoding of non pure upper.
+ */
+ if (dentry == dentry->d_sb->s_root)
+ origin = NULL;
+
+ /*
+ * TODO: handle encoding of non pure upper connectable file handle.
* Parent and child may not be on the same layer, so encode
* connectable file handle as an array of self ovl_fh and
* parent ovl_fh (type OVL_FILEID_WITH_PARENT).
*/
- if (!ovl_is_pure_upper_or_root(dentry, connectable))
+ if (connectable && !ovl_is_pure_upper_or_root(dentry))
return FILEID_INVALID;
- fh = ovl_encode_fh(ovl_dentry_upper(dentry), true, connectable);
+ /*
+ * We can only encode upper with origin if it is indexed, so NFS export
+ * will work only if overlay was mounted with index=all from the start.
+ *
+ * TODO: Either create index from origin information at encode time
+ * or encode non-indexed origin inode. The latter option requires
+ * that both non-dir and dir inodes will be indexed on encode
+ * time if upper has been renamed/redirected and that on decode,
+ * when index is not found for decoded lower, lookup upper by name
+ * with same path as decoded lower, while looking for indexed
+ * renamed parent directories along the path.
+ */
+ if (upper && origin && !ovl_test_flag(OVL_INDEX, d_inode(dentry)))
+ return FILEID_INVALID;
+
+ /*
+ * Copy up directory on encode to create an index. We need the index
+ * to decode a connected upper dir dentry, which we will use to
+ * reconnect a disconnected overlay dir dentry.
+ *
+ * TODO: walk back lower parent chain on decode to reconnect overlay
+ * dir dentry until an indexed dir is found, then continue to
+ * walk back upper parrent chain.
+ */
+ if (d_is_dir(dentry) && !upper) {
+ int err;
+
+ if (ovl_want_write(dentry))
+ return FILEID_INVALID;
+
+ err = ovl_copy_up(dentry);
+
+ ovl_drop_write(dentry);
+ if (err)
+ return FILEID_INVALID;
+
+ upper = ovl_dentry_upper(dentry);
+ }
+
+ /*
+ * The real encoded inode is the same real inode that is used to hash
+ * the overlay inode, so we can find overlay inode when decoding the
+ * real file handle. For merge dir and non-dir with origin, encode the
+ * origin inode. For root dir and pure upper, encode the upper inode.
+ */
+ fh = ovl_encode_fh(origin ?: upper, !origin, connectable);
if (IS_ERR(fh))
return FILEID_INVALID;
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 10/14] ovl: obtain a non-pure-upper disconnected dentry
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (8 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 09/14] ovl: encode non-pure-upper non-connectable " Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 11/14] ovl: decode non-pure-upper non-connectable file handles Amir Goldstein
` (5 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Instantiate an overlay dentry from lower real inode and optionally
from upper real inode with origin xattr.
This is needed for decoding an overlay inode that was encoded by real
lower file handle and since then may or may not have been copied up
and indexed.
This implementation is currently limited to non-dir and merge dirs
with single lower dir. For this reason, NFS export support currently
requires overlay with single lower layer.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/export.c | 44 +++++++++++++++++++++++++++++-----------
fs/overlayfs/namei.c | 52 ++++++++++++++++++++++++++++++------------------
fs/overlayfs/overlayfs.h | 4 ++++
3 files changed, 69 insertions(+), 31 deletions(-)
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 7ebe04317647..476e2a74aca7 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -176,17 +176,13 @@ static struct dentry *ovl_obtain_alias(struct super_block *sb,
struct dentry *upper,
struct dentry *lower)
{
+ struct ovl_fs *ofs = sb->s_fs_info;
struct inode *inode;
struct dentry *dentry;
+ struct dentry *index = lower ? upper : NULL;
struct ovl_entry *oe;
- /* TODO: handle decoding of non pure upper */
- if (lower) {
- dput(upper);
- return ERR_PTR(-EINVAL);
- }
-
- inode = ovl_get_inode(sb, upper, NULL, NULL);
+ inode = ovl_get_inode(sb, upper, lower, index);
if (IS_ERR(inode)) {
dput(upper);
return ERR_CAST(inode);
@@ -206,16 +202,40 @@ static struct dentry *ovl_obtain_alias(struct super_block *sb,
return dentry;
}
- oe = ovl_alloc_entry(0);
+ /*
+ * This implementation is currently limited to non-dir and merge dirs
+ * with single lower dir. For this reason, NFS export support currently
+ * requires overlay with single lower layer.
+ *
+ * TODO: use ovl_lookup or derivative to populate lowerstack with more
+ * lower dirs to support NFS export with multi lower layers.
+ */
+ oe = ovl_alloc_entry(lower ? 1 : 0);
if (!oe) {
dput(dentry);
return ERR_PTR(-ENOMEM);
}
-
+ if (lower) {
+ oe->lowerstack->dentry = dget(lower);
+ oe->lowerstack->mnt = ofs->lower_mnt[0];
+ }
dentry->d_fsdata = oe;
- ovl_dentry_set_upper_alias(dentry);
- if (d_is_dir(upper) && ovl_is_opaquedir(upper))
- ovl_dentry_set_opaque(dentry);
+
+ if (upper) {
+ ovl_dentry_set_upper_alias(dentry);
+ if (d_is_dir(upper)) {
+ size_t len = 0;
+ char *redirect = ovl_get_redirect_xattr(upper, &len);
+
+ if (redirect)
+ ovl_dentry_set_redirect(dentry, redirect);
+ if (ovl_is_opaquedir(upper))
+ ovl_dentry_set_opaque(dentry);
+ }
+ }
+
+ if (index)
+ ovl_set_flag(OVL_INDEX, inode);
return dentry;
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 4f20af22bd0b..796c869559a9 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -26,8 +26,7 @@ struct ovl_lookup_data {
char *redirect;
};
-static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
- size_t prelen, const char *post)
+char *ovl_get_redirect_xattr(struct dentry *dentry, size_t *len)
{
int res;
char *s, *next, *buf = NULL;
@@ -35,12 +34,12 @@ static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
res = vfs_getxattr(dentry, OVL_XATTR_REDIRECT, NULL, 0);
if (res < 0) {
if (res == -ENODATA || res == -EOPNOTSUPP)
- return 0;
+ return NULL;
goto fail;
}
- buf = kzalloc(prelen + res + strlen(post) + 1, GFP_KERNEL);
+ buf = kzalloc(res + *len + 1, GFP_KERNEL);
if (!buf)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
if (res == 0)
goto invalid;
@@ -59,22 +58,14 @@ static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
} else {
if (strchr(buf, '/') != NULL)
goto invalid;
-
- memmove(buf + prelen, buf, res);
- memcpy(buf, d->name.name, prelen);
}
- strcat(buf, post);
- kfree(d->redirect);
- d->redirect = buf;
- d->name.name = d->redirect;
- d->name.len = strlen(d->redirect);
-
- return 0;
+ *len = res;
+ return buf;
err_free:
kfree(buf);
- return 0;
+ return NULL;
fail:
pr_warn_ratelimited("overlayfs: failed to get redirect (%i)\n", res);
goto err_free;
@@ -83,6 +74,29 @@ static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
goto err_free;
}
+static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
+ size_t prelen, const char *post)
+{
+ size_t len = prelen + strlen(post);
+ char *buf = ovl_get_redirect_xattr(dentry, &len);
+
+ if (IS_ERR_OR_NULL(buf))
+ return PTR_ERR(buf);
+
+ if (buf[0] != '/' && prelen) {
+ memmove(buf + prelen, buf, len);
+ memcpy(buf, d->name.name, prelen);
+ }
+
+ strcat(buf, post);
+ kfree(d->redirect);
+ d->redirect = buf;
+ d->name.name = d->redirect;
+ d->name.len = strlen(d->redirect);
+
+ return 0;
+}
+
static int ovl_acceptable(void *mnt, struct dentry *dentry)
{
/*
@@ -348,9 +362,9 @@ static int ovl_check_origin_fh(struct ovl_fh *fh, struct dentry *upperdentry,
return -ESTALE;
}
-static int ovl_check_origin(struct dentry *upperdentry,
- struct path *lowerstack, unsigned int numlower,
- struct path **stackp, unsigned int *ctrp)
+int ovl_check_origin(struct dentry *upperdentry, struct path *lowerstack,
+ unsigned int numlower, struct path **stackp,
+ unsigned int *ctrp)
{
struct ovl_fh *fh = ovl_get_origin_fh(upperdentry);
int err;
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 75a8b10d4e10..ffc5a955478e 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -271,8 +271,12 @@ static inline bool ovl_is_opaquedir(struct dentry *dentry)
/* namei.c */
+char *ovl_get_redirect_xattr(struct dentry *dentry, size_t *len);
int ovl_check_fh_len(struct ovl_fh *fh, int fh_len);
struct dentry *ovl_decode_fh(struct ovl_fh *fh, struct vfsmount *mnt);
+int ovl_check_origin(struct dentry *upperdentry, struct path *lowerstack,
+ unsigned int numlower, struct path **stackp,
+ unsigned int *ctrp);
int ovl_verify_origin(struct dentry *dentry, struct dentry *origin,
bool is_upper, bool set);
int ovl_verify_index(struct dentry *index, struct vfsmount *mnt,
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 11/14] ovl: decode non-pure-upper non-connectable file handles
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (9 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 10/14] ovl: obtain a non-pure-upper disconnected dentry Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 12/14] ovl: reconnect non-pure-upper dir " Amir Goldstein
` (4 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
When decoding an overlay inode that was encoded by real lower file
handle, we first decode the lower file handle, then we check if that
lower inode has been copied up and indexed.
If index is not found, we try to obtain a non-upper overlay dentry.
If index is found, we get the upper inode from index and try to obtain
an upper overlay dentry with lower origin.
If a whiteout index is found, overlay file handle is treated as stale.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/export.c | 83 ++++++++++++++++++++++++++++++++++++++++--------
fs/overlayfs/namei.c | 21 ++++++++----
fs/overlayfs/overlayfs.h | 6 +++-
3 files changed, 88 insertions(+), 22 deletions(-)
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 476e2a74aca7..f97f68e92eba 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -241,25 +241,14 @@ static struct dentry *ovl_obtain_alias(struct super_block *sb,
}
-static struct dentry *ovl_fh_to_d(struct super_block *sb, struct fid *fid,
- int fh_len, int fh_type, bool to_parent)
+static struct dentry *ovl_upper_fh_to_d(struct super_block *sb,
+ struct ovl_fh *fh, bool to_parent)
{
struct ovl_fs *ofs = sb->s_fs_info;
struct vfsmount *mnt = ofs->upper_mnt;
struct dentry *upper;
- struct ovl_fh *fh = (struct ovl_fh *) fid;
- int err;
-
- /* TODO: handle file handle with parent from different layer */
- if (fh_type != OVL_FILEID_WITHOUT_PARENT)
- return ERR_PTR(-EINVAL);
-
- err = ovl_check_fh_len(fh, fh_len << 2);
- if (err)
- return ERR_PTR(err);
- /* TODO: handle decoding of non pure upper */
- if (!mnt || !(fh->flags & OVL_FH_FLAG_PATH_UPPER))
+ if (!mnt)
return NULL;
upper = ovl_decode_fh(fh, mnt);
@@ -288,6 +277,72 @@ static struct dentry *ovl_fh_to_d(struct super_block *sb, struct fid *fid,
return ovl_obtain_alias(sb, upper, NULL);
}
+static struct dentry *ovl_fh_to_d(struct super_block *sb, struct fid *fid,
+ int fh_len, int fh_type, bool to_parent)
+{
+ struct ovl_fs *ofs = sb->s_fs_info;
+ struct dentry *upper = NULL;
+ struct dentry *index = NULL;
+ struct dentry *origin = NULL;
+ struct dentry *dentry = NULL;
+ struct ovl_fh *fh = (struct ovl_fh *) fid;
+ int err, i;
+
+ /* TODO: handle file handle with parent from different layer */
+ if (fh_type != OVL_FILEID_WITHOUT_PARENT)
+ return ERR_PTR(-EINVAL);
+
+ err = ovl_check_fh_len(fh, fh_len << 2);
+ if (err)
+ return ERR_PTR(err);
+
+ if (fh->flags & OVL_FH_FLAG_PATH_UPPER)
+ return ovl_upper_fh_to_d(sb, fh, to_parent);
+
+ /* TODO: decode parent from fh_type OVL_FILEID_WITH_PARENT */
+ if (to_parent)
+ return ERR_PTR(-EINVAL);
+
+ /* Find lower layer by UUID and decode */
+ for (i = 0; i < ofs->numlower; i++) {
+ origin = ovl_decode_fh(fh, ofs->lower_mnt[i]);
+ if (origin)
+ break;
+ }
+
+ if (IS_ERR_OR_NULL(origin))
+ return origin;
+
+ /* Lookup index by decoded origin */
+ index = ovl_lookup_index(ofs->indexdir, NULL, origin);
+ if (IS_ERR(index)) {
+ dput(origin);
+ return index;
+ }
+ if (index) {
+ /* Get upper dentry from index */
+ upper = ovl_index_upper(index, ofs->upper_mnt);
+ err = PTR_ERR(upper);
+ if (IS_ERR(upper))
+ goto out_err;
+
+ err = ovl_verify_origin(upper, origin, false, false);
+ if (err)
+ goto out_err;
+ }
+
+ dentry = ovl_obtain_alias(sb, upper, origin);
+
+out:
+ dput(index);
+ dput(origin);
+ return dentry;
+
+out_err:
+ dentry = ERR_PTR(err);
+ goto out;
+}
+
static struct dentry *ovl_fh_to_dentry(struct super_block *sb, struct fid *fid,
int fh_len, int fh_type)
{
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 796c869559a9..0f19c2cf43fc 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -609,11 +609,9 @@ int ovl_get_index_name(struct dentry *origin, struct qstr *name)
}
-static struct dentry *ovl_lookup_index(struct dentry *dentry,
- struct dentry *upper,
- struct dentry *origin)
+struct dentry *ovl_lookup_index(struct dentry *indexdir, struct dentry *upper,
+ struct dentry *origin)
{
- struct ovl_fs *ofs = dentry->d_sb->s_fs_info;
struct dentry *index;
struct inode *inode;
struct qstr name;
@@ -624,7 +622,7 @@ static struct dentry *ovl_lookup_index(struct dentry *dentry,
if (err)
return ERR_PTR(err);
- index = lookup_one_len_unlocked(name.name, ofs->indexdir, name.len);
+ index = lookup_one_len_unlocked(name.name, indexdir, name.len);
if (IS_ERR(index)) {
err = PTR_ERR(index);
if (err == -ENOENT) {
@@ -645,6 +643,15 @@ static struct dentry *ovl_lookup_index(struct dentry *dentry,
upper);
}
goto out_dput;
+ } else if (ovl_is_whiteout(index) && !upper) {
+ /*
+ * When index lookup is called with no upper for decoding an
+ * overlay file handle, a whiteout index implies that decode
+ * should treat file handle as stale.
+ */
+ dput(index);
+ index = ERR_PTR(-ESTALE);
+ goto out;
} else if (ovl_dentry_weird(index) || ovl_is_whiteout(index) ||
((inode->i_mode ^ d_inode(origin)->i_mode) & S_IFMT)) {
/*
@@ -870,8 +877,8 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
* TODO: update origin and index in case lower dir has changed and
* store new generation number xattr in index for NFS export.
*/
- if (ctr && ovl_indexdir(dentry->d_sb) && origin) {
- index = ovl_lookup_index(dentry, upperdentry, origin);
+ if (ctr && ofs->indexdir && origin) {
+ index = ovl_lookup_index(ofs->indexdir, upperdentry, origin);
if (IS_ERR(index)) {
err = PTR_ERR(index);
index = NULL;
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index ffc5a955478e..3a801511d163 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -279,11 +279,15 @@ int ovl_check_origin(struct dentry *upperdentry, struct path *lowerstack,
unsigned int *ctrp);
int ovl_verify_origin(struct dentry *dentry, struct dentry *origin,
bool is_upper, bool set);
+struct dentry *ovl_index_upper(struct dentry *index, struct vfsmount *mnt);
int ovl_verify_index(struct dentry *index, struct vfsmount *mnt,
struct path *lowerstack, unsigned int numlower);
int ovl_get_index_name(struct dentry *origin, struct qstr *name);
+struct dentry *ovl_lookup_index(struct dentry *indexdir, struct dentry *upper,
+ struct dentry *origin);
int ovl_path_next(int idx, struct dentry *dentry, struct path *path, int *idxp);
-struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, unsigned int flags);
+struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
+ unsigned int flags);
bool ovl_lower_positive(struct dentry *dentry);
/* readdir.c */
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 12/14] ovl: reconnect non-pure-upper dir file handles
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (10 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 11/14] ovl: decode non-pure-upper non-connectable file handles Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 13/14] ovl: wire up NFS export support Amir Goldstein
` (3 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Directory file handles must be connectable. With current implementation
directories are copied up on encode, so directory file handles are
guarantied to be upper.
Implement get_parent() operation to reconnect a non pure upper overlay dir
dentry. fh_to_parent() operation is not implemented for non pure upper, so
connectable non-dir non pure upper file handles are not supported.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/export.c | 58 ++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 51 insertions(+), 7 deletions(-)
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index f97f68e92eba..7c04ae99c18b 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -357,11 +357,14 @@ static struct dentry *ovl_fh_to_parent(struct super_block *sb, struct fid *fid,
static struct dentry *ovl_get_parent(struct dentry *dentry)
{
+ struct dentry *root = dentry->d_sb->s_root;
+ struct ovl_entry *roe = root->d_fsdata;
+ struct dentry *parent;
struct dentry *upper;
-
- /* TODO: handle connecting of non pure upper */
- if (ovl_dentry_lower(dentry))
- return ERR_PTR(-EACCES);
+ struct dentry *origin = NULL;
+ struct path *stack = NULL;
+ unsigned int ctr = 0;
+ int err;
/*
* When ovl_fh_to_d() returns an overlay dentry, its real upper
@@ -369,16 +372,57 @@ static struct dentry *ovl_get_parent(struct dentry *dentry)
* the upper dentry is done by ovl_decode_fh() when decoding the
* real upper file handle, so here we have the upper dentry parent
* and we need to instantiate an overlay dentry with upper dentry
- * parent.
+ * parent and the lower dir pointed to by origin xattr.
+ *
+ * TODO: handle reconnecting of non upper overlay dentry.
*/
upper = ovl_dentry_upper(dentry);
if (!upper || (upper->d_flags & DCACHE_DISCONNECTED))
return ERR_PTR(-ESTALE);
upper = dget_parent(upper);
+ if (upper == ovl_dentry_upper(root)) {
+ dput(upper);
+ return dget(root);
+ }
- /* Find or instantiate a pure upper dentry */
- return ovl_obtain_alias(dentry->d_sb, upper, NULL);
+ /* Check if parent is merge dir or pure upper */
+ err = ovl_check_origin(upper, roe->lowerstack, roe->numlower,
+ &stack, &ctr);
+ if (err)
+ goto out_err;
+
+ if (ctr) {
+ struct dentry *index;
+
+ /*
+ * Lookup index by decoded origin to verify dir is indexed.
+ * We only decode upper with origin if it is indexed, so NFS
+ * export will work only if overlay was mounted with index=all
+ * from the start.
+ */
+ origin = stack[0].dentry;
+ index = ovl_lookup_index(ovl_indexdir(dentry->d_sb), upper,
+ origin);
+ err = index ? PTR_ERR(index) : -ESTALE;
+ if (IS_ERR_OR_NULL(index))
+ goto out_err;
+
+ dput(index);
+ }
+
+ /* Find or instantiate an upper dentry */
+ parent = ovl_obtain_alias(dentry->d_sb, upper, origin);
+
+out:
+ dput(origin);
+ kfree(stack);
+ return parent;
+
+out_err:
+ dput(upper);
+ parent = ERR_PTR(err);
+ goto out;
}
const struct export_operations ovl_export_operations = {
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 13/14] ovl: wire up NFS export support
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (11 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 12/14] ovl: reconnect non-pure-upper dir " Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-17 16:44 ` [PATCH 14/14] ovl: document NFS export Amir Goldstein
` (2 subsequent siblings)
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Currently, overlayfs file handles decoding is guarantied to
work if all the following conditions are met:
1. All underlying layers support file handles
2. All lower files and directories are indexed on copy up (index=all)
3. A merge directory lower dir must match copy up origin (verify_dir)
4. A merge directory is limited to single lower layer
With all conditions met, enable NFS export support.
All but the first condition can be relaxed in the future with
some extra work.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
fs/overlayfs/super.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 3c98991eebcd..36d6ed20d496 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -1124,6 +1124,27 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
}
}
+ /*
+ * NFS export requires that all layers support file handles and
+ * that all files and dirs are indexed on copy up (index=all).
+ * We check that all layers support file handles for enabling index.
+ *
+ * A directory index has all the information to instantiate the
+ * encoded overlay directory dentry IFF a merge directory is limited
+ * to a single lower dir and the lower dir matches the copy up origin
+ * stored in upper dir (verify_dir).
+ *
+ * XXX: index=all can be relaxed by indexing on rename and looking
+ * up non-indexed overlay dentries by lower dentry path.
+ * verify_dir can be relaxed by updating index when mismatch
+ * lower merge dir is found and update index generation.
+ * single lower layer can be relaxed by looking up overlay
+ * directory dentries by lower or upper dentry path.
+ */
+ if (ufs->config.index == OVL_INDEX_ALL &&
+ ufs->config.verify_dir && ufs->numlower == 1)
+ sb->s_export_op = &ovl_export_operations;
+
if (remote)
sb->s_d_op = &ovl_reval_dentry_operations;
else
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 14/14] ovl: document NFS export
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (12 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 13/14] ovl: wire up NFS export support Amir Goldstein
@ 2017-10-17 16:44 ` Amir Goldstein
2017-10-18 18:43 ` [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
2017-11-09 19:02 ` J . Bruce Fields
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-17 16:44 UTC (permalink / raw)
To: Miklos Szeredi
Cc: Jeff Layton, J . Bruce Fields, linux-unionfs, linux-fsdevel
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
Documentation/filesystems/overlayfs.txt | 61 +++++++++++++++++++++++++++++++++
1 file changed, 61 insertions(+)
diff --git a/Documentation/filesystems/overlayfs.txt b/Documentation/filesystems/overlayfs.txt
index d89041baceb9..dbd829b0b12b 100644
--- a/Documentation/filesystems/overlayfs.txt
+++ b/Documentation/filesystems/overlayfs.txt
@@ -285,6 +285,67 @@ merged with the upper directory instead. The "verify_dir" feature,
therefore, makes an overlay mount cope better with the situations of
lower directory rename and delete.
+
+NFS export
+----------
+
+An overlay filesystem may be exported to NFS when:
+- All underlying filesystems support NFS file handles
+- overlay is mounted with the mount options "index=all,verify_dir"
+- overlay is not using multiple lower layers
+
+With the "index=all" feature, on copy_up of any lower object, an index
+entry is created under the index directory. The index entry name is the
+hexadecimal representation of the copy up origin file handle. For a
+non-directory object, the index entry is a hard link to the upper inode.
+For a directory object, the index entry has an extended attribute
+"trusted.overlay.origin" with an encoded file handle of the upper
+directory inode.
+
+When encoding a file handle from an overlay filesystem object, the
+following rules apply:
+
+1. For a non-upper object, encode a lower file handle from lower inode
+2. For a pure-upper object, encode an upper file handle from upper inode
+3. For an indexed object, encode a lower file handle from copy_up origin
+
+Encoding a lower directory file handle will copy up and index that
+directory, so the index can be use on decode to find the upper directory.
+An existing non-indexed merged directory cannot be encoded with current
+NFS export implementation.
+
+The encoded overlay file handle includes:
+ - Header including path type information (e.g. lower/upper)
+ - UUID of the underlying filesystem
+ - Underlying filesystem encoding of underlying inode
+
+This encoding is identical to the encoding of copy_up origin stored in
+"trusted.overlay.origin".
+
+When decoding an overlay file handle, the following steps are followed:
+
+1. Find underlying layer by UUID and path type information.
+2. Decode the underlying filesystem file handle to underlying dentry.
+3. Lookup the file handle in index directory by name.
+4. If not found in index, instantiate an overlay dentry from the decoded
+ lower dentry.
+6. If a non-directory is found in index, instantiate a disconnected overlay
+ dentry from the upper index dentry and the decoded lower dentry.
+5. If a directory is found in index, decode upper dentry from file handle
+ stored in extended attribute "trusted.overlay.origin" in the index and
+ instantiate an overlay dentry from the decoded upper and lower dentries.
+7. If a whiteout is found in index, return ESTALE. This represents an
+ overlay object that was deleted after its file handle was encoded.
+
+The "verify_dir" mount option and no multiple lower layer requirements,
+ensure that a decoded overlay directory object will be equivalent to the
+object that was used to encode the file handle.
+
+Decoding a non-directory file handle may return a disconnected dentry.
+copy_up of that disconnected dentry will create an upper index entry with
+no upper alias.
+
+
Testsuite
---------
--
2.7.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH 00/14] Overlayfs NFS export support
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (13 preceding siblings ...)
2017-10-17 16:44 ` [PATCH 14/14] ovl: document NFS export Amir Goldstein
@ 2017-10-18 18:43 ` Amir Goldstein
2017-11-09 19:02 ` J . Bruce Fields
15 siblings, 0 replies; 22+ messages in thread
From: Amir Goldstein @ 2017-10-18 18:43 UTC (permalink / raw)
To: Miklos Szeredi; +Cc: Jeff Layton, J . Bruce Fields, overlayfs, linux-fsdevel
On Tue, Oct 17, 2017 at 7:44 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> Miklos,
>
> This series implements NFS export support [3] and is based on
> two prep patch sets [1][2] posted earlier to overlayfs list.
> NFS export is enabled for overlayfs mount with the 'verify_dir'
> and 'index=all' mount options.
>
> The current implementation will copy up and index directories when
> directory file handles are encoded. Those extra copy ups could be
> avoided in the future.
>
> The current implementation does NOT support encoding connectable
> non-dir file handles for all overlay path types, so overlayfs should
> not be exported with the 'subtree_check' option.
> I hope this is not a show stopper for merging NFS export support?
FYI, I implemented encoding of type OVL_FILEID_WITH_PARENT and
pushed 2 more patches to branch ovl-nfs-export-v1, so now export with
'subtree_check' should work fine.
I used a test patch (at the tip of ovl-nfs-export-v1) to encode connectable
file handles from name_to_handle_at() to unit test this code with my xfstests.
>
> You'll notice that the series start by implementing naiive support
> for pure upper files export and later on, replaces the implementation
> of some of the operations. This may seem strange, but I found it
> convenient to progress slowly between testable mile stones and I find
> the series easier to review this way. Let me know if you want me to
> change the way that this series is organized to get rid of code that
> is later removed.
The last two "non-pure-upper connectable file handles" patches
remove even more of the pure-upper specific implementation code,
so I should definitely rework the series for v2 to discard the temporary
pure upper implementation patches.
>
> To unit test overlayfs file handles, I enhanced xfstest open_by_handle
> test utility to encode/decode directories and check several other cases
> that were not covered by the original xfstest test. On my xfstests NFS
> export branch [4], there are two generic tests and one overlayfs specific
> test on the 'exportfs' group.
>
> I also ran the NFSTest nfstest_posix group on an exported overlayfs
> mount, but that test only creates pure upper files in overlay upper dir,
> so it is not much of a stress to the implementation.
>
> Amir.
>
> [1] https://github.com/amir73il/linux/commits/ovl-verify-dir
> [2] https://github.com/amir73il/linux/commits/ovl-index-all
> [3] https://github.com/amir73il/linux/commits/ovl-nfs-export
> [4] https://github.com/amir73il/xfstests/commits/ovl-nfs-export
>
> Amir Goldstein (14):
> ovl: hash all overlay inodes for NFS export
> ovl: grab i_count reference of lower inode
> ovl: use d_splice_alias() in place of d_add() in lookup
> ovl: copy up of disconnected dentries
> ovl: encode/decode pure-upper non-connectable file handles
> ovl: encode pure-upper connectable file handles
> ovl: decode pure-upper connectable file handles
> ovl: encode/decode struct ovl_fh format file handles
> ovl: encode non-pure-upper non-connectable file handles
> ovl: obtain a non-pure-upper disconnected dentry
> ovl: decode non-pure-upper non-connectable file handles
> ovl: reconnect non-pure-upper dir file handles
> ovl: wire up NFS export support
> ovl: document NFS export
>
> Documentation/filesystems/overlayfs.txt | 61 +++++
> fs/overlayfs/Makefile | 3 +-
> fs/overlayfs/copy_up.c | 74 ++++--
> fs/overlayfs/export.c | 433 ++++++++++++++++++++++++++++++++
> fs/overlayfs/inode.c | 23 +-
> fs/overlayfs/namei.c | 96 ++++---
> fs/overlayfs/overlayfs.h | 32 ++-
> fs/overlayfs/super.c | 25 ++
> fs/overlayfs/util.c | 13 +-
> 9 files changed, 678 insertions(+), 82 deletions(-)
> create mode 100644 fs/overlayfs/export.c
>
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 00/14] Overlayfs NFS export support
2017-10-17 16:44 [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
` (14 preceding siblings ...)
2017-10-18 18:43 ` [PATCH 00/14] Overlayfs NFS export support Amir Goldstein
@ 2017-11-09 19:02 ` J . Bruce Fields
2017-11-09 19:20 ` Jeff Layton
2017-11-09 19:59 ` Amir Goldstein
15 siblings, 2 replies; 22+ messages in thread
From: J . Bruce Fields @ 2017-11-09 19:02 UTC (permalink / raw)
To: Amir Goldstein; +Cc: Miklos Szeredi, Jeff Layton, linux-unionfs, linux-fsdevel
On Tue, Oct 17, 2017 at 07:44:17PM +0300, Amir Goldstein wrote:
> Miklos,
>
> This series implements NFS export support [3] and is based on
> two prep patch sets [1][2] posted earlier to overlayfs list.
> NFS export is enabled for overlayfs mount with the 'verify_dir'
> and 'index=all' mount options.
>
> The current implementation will copy up and index directories when
> directory file handles are encoded. Those extra copy ups could be
> avoided in the future.
>
> The current implementation does NOT support encoding connectable
> non-dir file handles for all overlay path types, so overlayfs should
> not be exported with the 'subtree_check' option.
> I hope this is not a show stopper for merging NFS export support?
I'm not a big fan of subtree_check, and I'd be OK with filesystems
opting out of support for it.
I'm not sure what to expect when you actually try to export overlayfs
with subtree_check. Does it fail on the export, or when clients first
try to access it, and what kind of error do you get?
> You'll notice that the series start by implementing naiive support
> for pure upper files export and later on, replaces the implementation
> of some of the operations. This may seem strange, but I found it
> convenient to progress slowly between testable mile stones and I find
> the series easier to review this way. Let me know if you want me to
> change the way that this series is organized to get rid of code that
> is later removed.
>
> To unit test overlayfs file handles, I enhanced xfstest open_by_handle
> test utility to encode/decode directories and check several other cases
> that were not covered by the original xfstest test. On my xfstests NFS
> export branch [4], there are two generic tests and one overlayfs specific
> test on the 'exportfs' group.
>
> I also ran the NFSTest nfstest_posix group on an exported overlayfs
> mount, but that test only creates pure upper files in overlay upper dir,
> so it is not much of a stress to the implementation.
Might also be interesting to run some nfs tests in a loop while
restarting the server?
--b.
>
> Amir.
>
> [1] https://github.com/amir73il/linux/commits/ovl-verify-dir
> [2] https://github.com/amir73il/linux/commits/ovl-index-all
> [3] https://github.com/amir73il/linux/commits/ovl-nfs-export
> [4] https://github.com/amir73il/xfstests/commits/ovl-nfs-export
>
> Amir Goldstein (14):
> ovl: hash all overlay inodes for NFS export
> ovl: grab i_count reference of lower inode
> ovl: use d_splice_alias() in place of d_add() in lookup
> ovl: copy up of disconnected dentries
> ovl: encode/decode pure-upper non-connectable file handles
> ovl: encode pure-upper connectable file handles
> ovl: decode pure-upper connectable file handles
> ovl: encode/decode struct ovl_fh format file handles
> ovl: encode non-pure-upper non-connectable file handles
> ovl: obtain a non-pure-upper disconnected dentry
> ovl: decode non-pure-upper non-connectable file handles
> ovl: reconnect non-pure-upper dir file handles
> ovl: wire up NFS export support
> ovl: document NFS export
>
> Documentation/filesystems/overlayfs.txt | 61 +++++
> fs/overlayfs/Makefile | 3 +-
> fs/overlayfs/copy_up.c | 74 ++++--
> fs/overlayfs/export.c | 433 ++++++++++++++++++++++++++++++++
> fs/overlayfs/inode.c | 23 +-
> fs/overlayfs/namei.c | 96 ++++---
> fs/overlayfs/overlayfs.h | 32 ++-
> fs/overlayfs/super.c | 25 ++
> fs/overlayfs/util.c | 13 +-
> 9 files changed, 678 insertions(+), 82 deletions(-)
> create mode 100644 fs/overlayfs/export.c
>
> --
> 2.7.4
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 00/14] Overlayfs NFS export support
2017-11-09 19:02 ` J . Bruce Fields
@ 2017-11-09 19:20 ` Jeff Layton
2017-11-09 19:59 ` Amir Goldstein
1 sibling, 0 replies; 22+ messages in thread
From: Jeff Layton @ 2017-11-09 19:20 UTC (permalink / raw)
To: J . Bruce Fields, Amir Goldstein
Cc: Miklos Szeredi, linux-unionfs, linux-fsdevel
On Thu, 2017-11-09 at 14:02 -0500, J . Bruce Fields wrote:
> On Tue, Oct 17, 2017 at 07:44:17PM +0300, Amir Goldstein wrote:
> > Miklos,
> >
> > This series implements NFS export support [3] and is based on
> > two prep patch sets [1][2] posted earlier to overlayfs list.
> > NFS export is enabled for overlayfs mount with the 'verify_dir'
> > and 'index=all' mount options.
> >
> > The current implementation will copy up and index directories when
> > directory file handles are encoded. Those extra copy ups could be
> > avoided in the future.
> >
> > The current implementation does NOT support encoding connectable
> > non-dir file handles for all overlay path types, so overlayfs should
> > not be exported with the 'subtree_check' option.
> > I hope this is not a show stopper for merging NFS export support?
>
> I'm not a big fan of subtree_check, and I'd be OK with filesystems
> opting out of support for it.
>
> I'm not sure what to expect when you actually try to export overlayfs
> with subtree_check. Does it fail on the export, or when clients first
> try to access it, and what kind of error do you get?
>
As part of the NFS reexport series that I posted a couple of years ago,
there was a set of patches that would make exportfs return a (sensible)
error message when it tried to enable it on a filesystem that didn't
support it.
We could look at resurrecting that if there is interest in allowing
filesystems to opt out of subtree checking. It wasn't terribly complex,
IIRC.
> > You'll notice that the series start by implementing naiive support
> > for pure upper files export and later on, replaces the implementation
> > of some of the operations. This may seem strange, but I found it
> > convenient to progress slowly between testable mile stones and I find
> > the series easier to review this way. Let me know if you want me to
> > change the way that this series is organized to get rid of code that
> > is later removed.
> >
> > To unit test overlayfs file handles, I enhanced xfstest open_by_handle
> > test utility to encode/decode directories and check several other cases
> > that were not covered by the original xfstest test. On my xfstests NFS
> > export branch [4], there are two generic tests and one overlayfs specific
> > test on the 'exportfs' group.
> >
> > I also ran the NFSTest nfstest_posix group on an exported overlayfs
> > mount, but that test only creates pure upper files in overlay upper dir,
> > so it is not much of a stress to the implementation.
>
> Might also be interesting to run some nfs tests in a loop while
> restarting the server?
>
> --b.
>
> >
> > Amir.
> >
> > [1] https://github.com/amir73il/linux/commits/ovl-verify-dir
> > [2] https://github.com/amir73il/linux/commits/ovl-index-all
> > [3] https://github.com/amir73il/linux/commits/ovl-nfs-export
> > [4] https://github.com/amir73il/xfstests/commits/ovl-nfs-export
> >
> > Amir Goldstein (14):
> > ovl: hash all overlay inodes for NFS export
> > ovl: grab i_count reference of lower inode
> > ovl: use d_splice_alias() in place of d_add() in lookup
> > ovl: copy up of disconnected dentries
> > ovl: encode/decode pure-upper non-connectable file handles
> > ovl: encode pure-upper connectable file handles
> > ovl: decode pure-upper connectable file handles
> > ovl: encode/decode struct ovl_fh format file handles
> > ovl: encode non-pure-upper non-connectable file handles
> > ovl: obtain a non-pure-upper disconnected dentry
> > ovl: decode non-pure-upper non-connectable file handles
> > ovl: reconnect non-pure-upper dir file handles
> > ovl: wire up NFS export support
> > ovl: document NFS export
> >
> > Documentation/filesystems/overlayfs.txt | 61 +++++
> > fs/overlayfs/Makefile | 3 +-
> > fs/overlayfs/copy_up.c | 74 ++++--
> > fs/overlayfs/export.c | 433 ++++++++++++++++++++++++++++++++
> > fs/overlayfs/inode.c | 23 +-
> > fs/overlayfs/namei.c | 96 ++++---
> > fs/overlayfs/overlayfs.h | 32 ++-
> > fs/overlayfs/super.c | 25 ++
> > fs/overlayfs/util.c | 13 +-
> > 9 files changed, 678 insertions(+), 82 deletions(-)
> > create mode 100644 fs/overlayfs/export.c
> >
> > --
> > 2.7.4
--
Jeff Layton <jlayton@kernel.org>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 00/14] Overlayfs NFS export support
2017-11-09 19:02 ` J . Bruce Fields
2017-11-09 19:20 ` Jeff Layton
@ 2017-11-09 19:59 ` Amir Goldstein
2017-11-09 21:55 ` J . Bruce Fields
1 sibling, 1 reply; 22+ messages in thread
From: Amir Goldstein @ 2017-11-09 19:59 UTC (permalink / raw)
To: J . Bruce Fields; +Cc: Miklos Szeredi, Jeff Layton, overlayfs, linux-fsdevel
On Thu, Nov 9, 2017 at 9:02 PM, J . Bruce Fields <bfields@fieldses.org> wrote:
> On Tue, Oct 17, 2017 at 07:44:17PM +0300, Amir Goldstein wrote:
>> Miklos,
>>
>> This series implements NFS export support [3] and is based on
>> two prep patch sets [1][2] posted earlier to overlayfs list.
>> NFS export is enabled for overlayfs mount with the 'verify_dir'
>> and 'index=all' mount options.
>>
>> The current implementation will copy up and index directories when
>> directory file handles are encoded. Those extra copy ups could be
>> avoided in the future.
>>
>> The current implementation does NOT support encoding connectable
>> non-dir file handles for all overlay path types, so overlayfs should
>> not be exported with the 'subtree_check' option.
>> I hope this is not a show stopper for merging NFS export support?
>
> I'm not a big fan of subtree_check, and I'd be OK with filesystems
> opting out of support for it.
>
> I'm not sure what to expect when you actually try to export overlayfs
> with subtree_check. Does it fail on the export, or when clients first
> try to access it, and what kind of error do you get?
Not an issue anymore.
I replied one day after this email that:
FYI, I implemented encoding of type OVL_FILEID_WITH_PARENT and
pushed 2 more patches to branch ovl-nfs-export-v1, so now export with
'subtree_check' should work fine.
I used a test patch (at the tip of ovl-nfs-export-v1) to encode connectable
file handles from name_to_handle_at() to unit test this code with my xfstests.
>
>> You'll notice that the series start by implementing naiive support
>> for pure upper files export and later on, replaces the implementation
>> of some of the operations. This may seem strange, but I found it
>> convenient to progress slowly between testable mile stones and I find
>> the series easier to review this way. Let me know if you want me to
>> change the way that this series is organized to get rid of code that
>> is later removed.
>>
>> To unit test overlayfs file handles, I enhanced xfstest open_by_handle
>> test utility to encode/decode directories and check several other cases
>> that were not covered by the original xfstest test. On my xfstests NFS
>> export branch [4], there are two generic tests and one overlayfs specific
>> test on the 'exportfs' group.
>>
>> I also ran the NFSTest nfstest_posix group on an exported overlayfs
>> mount, but that test only creates pure upper files in overlay upper dir,
>> so it is not much of a stress to the implementation.
>
> Might also be interesting to run some nfs tests in a loop while
> restarting the server?
>
I can try that, but I also need to create some setups with files in
lower dir and merge dirs and run tests on those, i.e. not only
files that the test creates.
BTW, the patches passed all the tests I wrote, I rebased them
on latest overlayfs-next, but still need to cleanup the series before
I post V2.
Thanks,
Amir.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 00/14] Overlayfs NFS export support
2017-11-09 19:59 ` Amir Goldstein
@ 2017-11-09 21:55 ` J . Bruce Fields
0 siblings, 0 replies; 22+ messages in thread
From: J . Bruce Fields @ 2017-11-09 21:55 UTC (permalink / raw)
To: Amir Goldstein; +Cc: Miklos Szeredi, Jeff Layton, overlayfs, linux-fsdevel
On Thu, Nov 09, 2017 at 09:59:10PM +0200, Amir Goldstein wrote:
> On Thu, Nov 9, 2017 at 9:02 PM, J . Bruce Fields <bfields@fieldses.org> wrote:
> > On Tue, Oct 17, 2017 at 07:44:17PM +0300, Amir Goldstein wrote:
> >> Miklos,
> >>
> >> This series implements NFS export support [3] and is based on
> >> two prep patch sets [1][2] posted earlier to overlayfs list.
> >> NFS export is enabled for overlayfs mount with the 'verify_dir'
> >> and 'index=all' mount options.
> >>
> >> The current implementation will copy up and index directories when
> >> directory file handles are encoded. Those extra copy ups could be
> >> avoided in the future.
> >>
> >> The current implementation does NOT support encoding connectable
> >> non-dir file handles for all overlay path types, so overlayfs should
> >> not be exported with the 'subtree_check' option.
> >> I hope this is not a show stopper for merging NFS export support?
> >
> > I'm not a big fan of subtree_check, and I'd be OK with filesystems
> > opting out of support for it.
> >
> > I'm not sure what to expect when you actually try to export overlayfs
> > with subtree_check. Does it fail on the export, or when clients first
> > try to access it, and what kind of error do you get?
>
> Not an issue anymore.
> I replied one day after this email that:
>
> FYI, I implemented encoding of type OVL_FILEID_WITH_PARENT and
> pushed 2 more patches to branch ovl-nfs-export-v1, so now export with
> 'subtree_check' should work fine.
Oops, sorry, OK.
--b.
^ permalink raw reply [flat|nested] 22+ messages in thread