[PATCH v3 00/16] overlayfs constant inode numbers

[PATCH v3 00/16] overlayfs constant inode numbers

[Amir Goldstein]

Miklos,

This is v3 series of redirect by file handle and constant inode numbers.
this series fixes constant inode numbers for stat(2) with any layer
configuration.

For the case of all *lower* layers on same fs that supports NFS export,
redirect by file handle will be used to optimize the lookup of the copy
up origin of non-dir inode.

For the case of *all* layers on same fs, overlayfs also gains:
- Persistent inode numbers for directories
- Consistent st_dev for all overlay objects

Consistcy of stat(2) st_ino with readdir(3) d_ino is NOT addressed by
this series. It will be addressed for the 'samefs' configuration by the
follow up 'stable inode' work, which is also going to address preserving
hardlinks on copy up.

This series is available for testing on [1].
unionmount-testsuite was instrumented to verify constant inode numbers
after rename/link and mount cycle [2].

Tested the following layer configurations:
 ./run --ov{,=0,=1} {,--samefs}

The following test fails the constant inode number verification of upper
hardlink in the non-same-lower-fs configuration:
 ./run --ov=1 hard-link

xfstest overlay/017 was added a check for constant and persistent inode
numbers across rename and mount cycle [3].
Tested also ./check -overlay -g quick.

Changes since v2:
- Storing uuid of lower origin (and root fh for now)
- Re-factor ovl_lookup_single() (per your comment)
- Verify uuid of same_lower_sb (allow zero uuid)
- Find layer by root fh (see how you feel about that)
- d_drop() ovl_link() target (please comment on that)
- Updated overlayfs.txt

Amir.

[1] https://github.com/amir73il/linux/commits/ovl-constino
[2] https://github.com/amir73il/unionmount-testsuite/commits/overlayfs-devel
[3] https://github.com/amir73il/xfstests/commits/overlayfs-devel

Amir Goldstein (16):
  ovl: store path type in dentry
  ovl: cram opaque boolean into type flags
  ovl: check if all layers are on the same fs
  ovl: store file handle of lower inode on copy up
  ovl: use an auxiliary var for overlay root entry
  ovl: factor out ovl_lookup_data()
  ovl: lookup redirect by file handle
  ovl: validate lower layer uuid and root on redirect by fh
  ovl: lookup non-dir inode copy up origin
  ovl: set the COPYUP type flag for non-dirs
  ovl: redirect non-dir by path on rename
  ovl: constant st_ino/st_dev across copy up
  ovl: persistent inode number for directories
  ovl: fix du --one-file-system on overlay mount
  ovl: persistent inode numbers for hardlinks
  ovl: update documentation w.r.t. constant inode numbers

 Documentation/filesystems/overlayfs.txt |  19 +-
 fs/overlayfs/copy_up.c                  | 142 +++++++++++++++
 fs/overlayfs/dir.c                      |  31 +++-
 fs/overlayfs/inode.c                    |  21 ++-
 fs/overlayfs/namei.c                    | 311 ++++++++++++++++++++++++++++----
 fs/overlayfs/overlayfs.h                |  37 ++++
 fs/overlayfs/ovl_entry.h                |   9 +-
 fs/overlayfs/super.c                    |  24 +++
 fs/overlayfs/util.c                     |  83 ++++++++-
 9 files changed, 622 insertions(+), 55 deletions(-)

-- 
2.7.4

[PATCH v3 01/16] ovl: store path type in dentry

[Amir Goldstein]

We would like to add more state info to ovl_entry soon (for const ino)
and this state info would be added as type flags.

Store the type value in ovl_entry and update the UPPER and MERGE type
flags when needed, so ovl_path_type() just returns the stored value.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c     |  1 +
 fs/overlayfs/overlayfs.h |  1 +
 fs/overlayfs/ovl_entry.h |  3 +++
 fs/overlayfs/super.c     |  1 +
 fs/overlayfs/util.c      | 30 ++++++++++++++++++++++++------
 5 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index b8b0778..8788fd7 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -338,6 +338,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	kfree(stack);
 	kfree(d.redirect);
 	dentry->d_fsdata = oe;
+	ovl_update_type(dentry, d.is_dir);
 	d_add(dentry, inode);
 
 	return NULL;
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 741dc0b..e90a548 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -155,6 +155,7 @@ struct ovl_entry *ovl_alloc_entry(unsigned int numlower);
 bool ovl_dentry_remote(struct dentry *dentry);
 bool ovl_dentry_weird(struct dentry *dentry);
 enum ovl_path_type ovl_path_type(struct dentry *dentry);
+enum ovl_path_type ovl_update_type(struct dentry *dentry, bool is_dir);
 void ovl_path_upper(struct dentry *dentry, struct path *path);
 void ovl_path_lower(struct dentry *dentry, struct path *path);
 enum ovl_path_type ovl_path_real(struct dentry *dentry, struct path *path);
diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 59614fa..293be5f 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -31,6 +31,8 @@ struct ovl_fs {
 	wait_queue_head_t copyup_wq;
 };
 
+enum ovl_path_type;
+
 /* private information held for every overlayfs dentry */
 struct ovl_entry {
 	struct dentry *__upperdentry;
@@ -44,6 +46,7 @@ struct ovl_entry {
 		};
 		struct rcu_head rcu;
 	};
+	enum ovl_path_type __type;
 	unsigned numlower;
 	struct path lowerstack[];
 };
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index c072a0c..671bac0 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -961,6 +961,7 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	kfree(stack);
 
 	root_dentry->d_fsdata = oe;
+	ovl_update_type(root_dentry, true);
 
 	realinode = d_inode(ovl_dentry_real(root_dentry));
 	ovl_inode_init(d_inode(root_dentry), realinode, !!upperpath.dentry);
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 8a1e24a..bbd6074 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -70,21 +70,38 @@ bool ovl_dentry_weird(struct dentry *dentry)
 enum ovl_path_type ovl_path_type(struct dentry *dentry)
 {
 	struct ovl_entry *oe = dentry->d_fsdata;
-	enum ovl_path_type type = 0;
+	enum ovl_path_type type = oe->__type;
 
-	if (oe->__upperdentry) {
-		type = __OVL_PATH_UPPER;
+	/* Matches smp_wmb() in ovl_update_type() */
+	smp_rmb();
+	return type;
+}
+
+enum ovl_path_type ovl_update_type(struct dentry *dentry, bool is_dir)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+	enum ovl_path_type type = oe->__type;
 
+	/* Update UPPER/MERGE flags and preserve the rest */
+	type &= ~(__OVL_PATH_UPPER | __OVL_PATH_MERGE);
+	if (oe->__upperdentry) {
+		type |= __OVL_PATH_UPPER;
 		/*
-		 * Non-dir dentry can hold lower dentry from previous
-		 * location.
+		 * Non-dir dentry can hold lower dentry from before
+		 * copy-up.
 		 */
-		if (oe->numlower && d_is_dir(dentry))
+		if (oe->numlower && is_dir)
 			type |= __OVL_PATH_MERGE;
 	} else {
 		if (oe->numlower > 1)
 			type |= __OVL_PATH_MERGE;
 	}
+	/*
+	 * Make sure type is consistent with __upperdentry before making it
+	 * visible to ovl_path_type().
+	 */
+	smp_wmb();
+	oe->__type = type;
 	return type;
 }
 
@@ -220,6 +237,7 @@ void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry)
 	 */
 	smp_wmb();
 	oe->__upperdentry = upperdentry;
+	ovl_update_type(dentry, d_is_dir(dentry));
 }
 
 void ovl_inode_init(struct inode *inode, struct inode *realinode, bool is_upper)
-- 
2.7.4

[PATCH v3 02/16] ovl: cram opaque boolean into type flags

[Amir Goldstein]

We are going to add more state info to ovl_entry soon (for const ino)
and this state info would be added as type flags.

It makes sense to treat 'opaque' in a similar way, so instead of using
a boolean member in ovl_entry use a type bit to represent opaqueness.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c     | 9 +++++----
 fs/overlayfs/overlayfs.h | 2 ++
 fs/overlayfs/ovl_entry.h | 1 -
 fs/overlayfs/util.c      | 5 +++--
 4 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 8788fd7..d660177 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -224,7 +224,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	struct dentry *upperdir, *upperdentry = NULL;
 	unsigned int ctr = 0;
 	struct inode *inode = NULL;
-	bool upperopaque = false;
+	enum ovl_path_type type = 0;
 	char *upperredirect = NULL;
 	struct dentry *this;
 	unsigned int i;
@@ -261,7 +261,8 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 			if (d.redirect[0] == '/')
 				poe = dentry->d_sb->s_root->d_fsdata;
 		}
-		upperopaque = d.opaque;
+		if (d.opaque)
+			type |= __OVL_PATH_OPAQUE;
 	}
 
 	if (!d.stop && poe->numlower) {
@@ -331,7 +332,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	}
 
 	revert_creds(old_cred);
-	oe->opaque = upperopaque;
+	oe->__type = type;
 	oe->redirect = upperredirect;
 	oe->__upperdentry = upperdentry;
 	memcpy(oe->lowerstack, stack, sizeof(struct path) * ctr);
@@ -372,7 +373,7 @@ bool ovl_lower_positive(struct dentry *dentry)
 	 * whiteout.
 	 */
 	if (!dentry->d_inode)
-		return oe->opaque;
+		return OVL_TYPE_OPAQUE(oe->__type);
 
 	/* Negative upper -> positive lower */
 	if (!oe->__upperdentry)
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index e90a548..9420101 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -12,10 +12,12 @@
 enum ovl_path_type {
 	__OVL_PATH_UPPER	= (1 << 0),
 	__OVL_PATH_MERGE	= (1 << 1),
+	__OVL_PATH_OPAQUE	= (1 << 2),
 };
 
 #define OVL_TYPE_UPPER(type)	((type) & __OVL_PATH_UPPER)
 #define OVL_TYPE_MERGE(type)	((type) & __OVL_PATH_MERGE)
+#define OVL_TYPE_OPAQUE(type)	((type) & __OVL_PATH_OPAQUE)
 
 #define OVL_XATTR_PREFIX XATTR_TRUSTED_PREFIX "overlay."
 #define OVL_XATTR_OPAQUE OVL_XATTR_PREFIX "opaque"
diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 293be5f..12c4922 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -41,7 +41,6 @@ struct ovl_entry {
 		struct {
 			u64 version;
 			const char *redirect;
-			bool opaque;
 			bool copying;
 		};
 		struct rcu_head rcu;
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index bbd6074..88cfc2a 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -181,7 +181,8 @@ void ovl_set_dir_cache(struct dentry *dentry, struct ovl_dir_cache *cache)
 bool ovl_dentry_is_opaque(struct dentry *dentry)
 {
 	struct ovl_entry *oe = dentry->d_fsdata;
-	return oe->opaque;
+
+	return OVL_TYPE_OPAQUE(oe->__type);
 }
 
 bool ovl_dentry_is_whiteout(struct dentry *dentry)
@@ -193,7 +194,7 @@ void ovl_dentry_set_opaque(struct dentry *dentry)
 {
 	struct ovl_entry *oe = dentry->d_fsdata;
 
-	oe->opaque = true;
+	oe->__type |= __OVL_PATH_OPAQUE;
 }
 
 bool ovl_redirect_dir(struct super_block *sb)
-- 
2.7.4

[PATCH v3 03/16] ovl: check if all layers are on the same fs

[Amir Goldstein]

Some features can only work when all lower layers are on the same fs
and some features require that upper layer is also on the same fs.
Test those conditions during mount time, so features can check them later.

Add helper ovl_same_lower_sb() to return the common super block in case
all lower layers are on the same fs and helper ovl_same_sb() to return
the super block common to all layers.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/overlayfs.h |  2 ++
 fs/overlayfs/ovl_entry.h |  3 +++
 fs/overlayfs/super.c     |  9 +++++++++
 fs/overlayfs/util.c      | 14 ++++++++++++++
 4 files changed, 28 insertions(+)

diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 9420101..48d0dae 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -153,6 +153,8 @@ int ovl_want_write(struct dentry *dentry);
 void ovl_drop_write(struct dentry *dentry);
 struct dentry *ovl_workdir(struct dentry *dentry);
 const struct cred *ovl_override_creds(struct super_block *sb);
+struct super_block *ovl_same_lower_sb(struct super_block *sb);
+struct super_block *ovl_same_sb(struct super_block *sb);
 struct ovl_entry *ovl_alloc_entry(unsigned int numlower);
 bool ovl_dentry_remote(struct dentry *dentry);
 bool ovl_dentry_weird(struct dentry *dentry);
diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 12c4922..41708bf 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -29,6 +29,9 @@ struct ovl_fs {
 	const struct cred *creator_cred;
 	bool tmpfile;
 	wait_queue_head_t copyup_wq;
+	/* sb common to all (or all lower) layers */
+	struct super_block *same_lower_sb;
+	struct super_block *same_sb;
 };
 
 enum ovl_path_type;
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 671bac0..b8830ee 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -898,6 +898,7 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	ufs->lower_mnt = kcalloc(numlower, sizeof(struct vfsmount *), GFP_KERNEL);
 	if (ufs->lower_mnt == NULL)
 		goto out_put_workdir;
+
 	for (i = 0; i < numlower; i++) {
 		struct vfsmount *mnt = clone_private_mount(&stack[i]);
 
@@ -914,11 +915,19 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 
 		ufs->lower_mnt[ufs->numlower] = mnt;
 		ufs->numlower++;
+
+		/* Check if all lower layers are on same sb */
+		if (i == 0)
+			ufs->same_lower_sb = mnt->mnt_sb;
+		else if (ufs->same_lower_sb != mnt->mnt_sb)
+			ufs->same_lower_sb = NULL;
 	}
 
 	/* If the upper fs is nonexistent, we mark overlayfs r/o too */
 	if (!ufs->upper_mnt)
 		sb->s_flags |= MS_RDONLY;
+	else if (ufs->upper_mnt->mnt_sb == ufs->same_lower_sb)
+		ufs->same_sb = ufs->same_lower_sb;
 
 	if (remote)
 		sb->s_d_op = &ovl_reval_dentry_operations;
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 88cfc2a..e015bc3 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -41,6 +41,20 @@ const struct cred *ovl_override_creds(struct super_block *sb)
 	return override_creds(ofs->creator_cred);
 }
 
+struct super_block *ovl_same_lower_sb(struct super_block *sb)
+{
+	struct ovl_fs *ofs = sb->s_fs_info;
+
+	return ofs->same_lower_sb;
+}
+
+struct super_block *ovl_same_sb(struct super_block *sb)
+{
+	struct ovl_fs *ofs = sb->s_fs_info;
+
+	return ofs->same_sb;
+}
+
 struct ovl_entry *ovl_alloc_entry(unsigned int numlower)
 {
 	size_t size = offsetof(struct ovl_entry, lowerstack[numlower]);
-- 
2.7.4

[PATCH v3 04/16] ovl: store file handle of lower inode on copy up

[Amir Goldstein]

Sometimes it is interesting to know if an upper file is pure
upper or a copy up target, and if it is a copy up target, it
may be interesting to find the copy up origin.

This will be used to preserve lower inode numbers across copy up.

Store the lower inode file handle in upper inode extended attribute
overlay.origin.fh on copy up to use it later for these cases.
Store the lower layer root file handle and lower filesystem uuid in
overlay.origin.root and overlay.origin.uuid, to validate that we
are looking for the origin file in the original layer.

On failure to encode lower file handle, store an invalid 'null'
handle, so we can always use the overlay.origin.fh xattr to tell
between a copy up and a pure upper inode.

If lower fs does not support NFS export ops or if not all lower
layers are on the same fs, don't try to encode a lower file handle
and use the 'null' handle instead.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/copy_up.c   | 142 +++++++++++++++++++++++++++++++++++++++++++++++
 fs/overlayfs/overlayfs.h |  29 ++++++++++
 fs/overlayfs/ovl_entry.h |   2 +
 fs/overlayfs/super.c     |  14 +++++
 fs/overlayfs/util.c      |  14 +++++
 5 files changed, 201 insertions(+)

diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
index 906ea6c..7cc7aea 100644
--- a/fs/overlayfs/copy_up.c
+++ b/fs/overlayfs/copy_up.c
@@ -20,6 +20,8 @@
 #include <linux/namei.h>
 #include <linux/fdtable.h>
 #include <linux/ratelimit.h>
+#include <linux/mount.h>
+#include <linux/exportfs.h>
 #include "overlayfs.h"
 #include "ovl_entry.h"
 
@@ -232,6 +234,138 @@ int ovl_set_attr(struct dentry *upperdentry, struct kstat *stat)
 	return err;
 }
 
+static bool ovl_can_decode_fh(struct super_block *sb)
+{
+	return sb->s_export_op && sb->s_export_op->fh_to_dentry;
+}
+
+static struct ovl_fh *ovl_decode_fh(struct dentry *lower)
+{
+	struct ovl_fh *fh;
+	int fh_type, fh_len, dwords;
+	void *buf = NULL;
+	void *ret = NULL;
+	int buflen = MAX_HANDLE_SZ;
+	int err;
+
+	err = -EOPNOTSUPP;
+	/* Do not encode file handle if we cannot decode it later */
+	if (!ovl_can_decode_fh(lower->d_sb))
+		goto out_err;
+
+	err = -ENOMEM;
+	buf = kmalloc(buflen, GFP_TEMPORARY);
+	if (!buf)
+		goto out_err;
+
+	fh = buf;
+	dwords = (buflen - offsetof(struct ovl_fh, fid)) >> 2;
+	fh_type = exportfs_encode_fh(lower,
+				     (struct fid *)fh->fid,
+				     &dwords, 1);
+	fh_len = (dwords << 2) + offsetof(struct ovl_fh, fid);
+
+	err = -EOVERFLOW;
+	if (fh_len > buflen || fh_type <= 0 || fh_type == FILEID_INVALID)
+		goto out_err;
+
+	fh->version = OVL_FH_VERSION;
+	fh->magic = OVL_FH_MAGIC;
+	fh->type = fh_type;
+	fh->len = fh_len;
+
+	err = -ENOMEM;
+	ret = kmalloc(fh_len, GFP_KERNEL);
+	if (!ret)
+		goto out_err;
+
+	memcpy(ret, buf, fh_len);
+
+	kfree(buf);
+	return ret;
+
+out_err:
+	pr_warn_ratelimited("overlay: failed to get redirect fh (%i)\n", err);
+	kfree(buf);
+	kfree(ret);
+	return ERR_PTR(err);
+}
+
+static const struct ovl_fh null_fh = {
+	.version = OVL_FH_VERSION,
+	.magic = OVL_FH_MAGIC,
+	.type = FILEID_INVALID,
+	.len = sizeof(struct ovl_fh),
+};
+
+static int ovl_set_origin(struct dentry *dentry, struct dentry *upper)
+{
+	struct path lowerpath;
+	struct super_block *lower_sb;
+	const struct ovl_fh *fh = NULL;
+	const struct ovl_fh *rootfh = NULL;
+	int err;
+
+	ovl_path_lower(dentry, &lowerpath);
+	if (WARN_ON(!lowerpath.mnt))
+		return -EIO;
+
+	/*
+	 * Encoding a lower file handle where several layers are on the
+	 * same fs, require ecoding the layer root as well, because when
+	 * decoding the lower file handle we must provide the lowermnt.
+	 */
+	lower_sb = lowerpath.mnt->mnt_sb;
+	if (ovl_redirect_fh(dentry->d_sb) && ovl_can_decode_fh(lower_sb)) {
+		fh = ovl_decode_fh(lowerpath.dentry);
+		rootfh = ovl_decode_fh(lowerpath.mnt->mnt_root);
+	}
+	/*
+	 * On failure to encode lower fh, store an invalid 'null' fh, so
+	 * we can use the overlay.origin.fh xattr to distignuish between
+	 * a copy up and a pure upper inode.  If lower fs does not support
+	 * encoding fh, don't try to encode again (for any lower layer).
+	 */
+	err = 0;
+	if (IS_ERR_OR_NULL(fh)) {
+		err = PTR_ERR(fh);
+		fh = &null_fh;
+	}
+	if (IS_ERR_OR_NULL(rootfh)) {
+		if (err != -EOPNOTSUPP)
+			err = PTR_ERR(rootfh);
+		rootfh = NULL;
+	}
+	if (err == -EOPNOTSUPP) {
+		pr_warn("overlay: file handle not supported by lower - turning off redirect_fh\n");
+		ovl_clear_redirect_fh(dentry->d_sb);
+	}
+
+	err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_FH, fh, fh->len, 0);
+	if (err)
+		goto out_err;
+
+	if (rootfh) {
+		err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_ROOT, rootfh,
+				      rootfh->len, 0);
+	}
+	if (err)
+		goto out_err;
+
+	if (fh != &null_fh) {
+		err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_UUID,
+				      lower_sb->s_uuid,
+				      sizeof(lower_sb->s_uuid), 0);
+	}
+
+out_err:
+	if (fh != &null_fh)
+		kfree(fh);
+	return err;
+	if (rootfh != &null_fh)
+		kfree(rootfh);
+}
+
 static int ovl_copy_up_locked(struct dentry *workdir, struct dentry *upperdir,
 			      struct dentry *dentry, struct path *lowerpath,
 			      struct kstat *stat, const char *link,
@@ -316,6 +450,14 @@ static int ovl_copy_up_locked(struct dentry *workdir, struct dentry *upperdir,
 	if (err)
 		goto out_cleanup;
 
+	/*
+	 * Store identifier of lower inode in upper inode xattr to
+	 * allow lookup of the copy up origin inode.
+	 */
+	err = ovl_set_origin(dentry, temp);
+	if (err)
+		goto out_cleanup;
+
 	if (tmpfile)
 		err = ovl_do_link(temp, udir, upper, true);
 	else
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 48d0dae..2395dd7 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -22,6 +22,33 @@ enum ovl_path_type {
 #define OVL_XATTR_PREFIX XATTR_TRUSTED_PREFIX "overlay."
 #define OVL_XATTR_OPAQUE OVL_XATTR_PREFIX "opaque"
 #define OVL_XATTR_REDIRECT OVL_XATTR_PREFIX "redirect"
+/*
+ * The tuple origin.{fh,layer,uuid} is a universal unique identifier
+ * for a copy up origin, where:
+ * origin.fh	- exported file handle of the lower file
+ * origin.root	- exported file handle of the lower layer root
+ * origin.uuid	- uuid of the lower filesystem
+ *
+ * origin.{fh,root} are stored in format of a variable length binary blob
+ * with struct ovl_fh header (total blob size up to 20 bytes).
+ * uuid is stored in raw format (16 bytes) as published by sb->s_uuid.
+ */
+#define OVL_XATTR_ORIGIN_	OVL_XATTR_PREFIX "origin."
+#define OVL_XATTR_ORIGIN_FH	OVL_XATTR_ORIGIN_ "fh"
+#define OVL_XATTR_ORIGIN_ROOT	OVL_XATTR_ORIGIN_ "root"
+#define OVL_XATTR_ORIGIN_UUID	OVL_XATTR_ORIGIN_ "uuid"
+
+/* On-disk and in-memeory format for redirect by file handle */
+#define OVL_FH_VERSION	0
+#define OVL_FH_MAGIC	0xfb
+
+struct ovl_fh {
+	unsigned char version;	/* 0 */
+	unsigned char magic;	/* 0xfb */
+	unsigned char len;	/* size of this header + size of fid */
+	unsigned char type;	/* fid_type of fid */
+	unsigned char fid[0];	/* file identifier */
+} __packed;
 
 #define OVL_ISUPPER_MASK 1UL
 
@@ -175,6 +202,8 @@ bool ovl_redirect_dir(struct super_block *sb);
 void ovl_clear_redirect_dir(struct super_block *sb);
 const char *ovl_dentry_get_redirect(struct dentry *dentry);
 void ovl_dentry_set_redirect(struct dentry *dentry, const char *redirect);
+bool ovl_redirect_fh(struct super_block *sb);
+void ovl_clear_redirect_fh(struct super_block *sb);
 void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry);
 void ovl_inode_init(struct inode *inode, struct inode *realinode,
 		    bool is_upper);
diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 41708bf..2172dc5 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -32,6 +32,8 @@ struct ovl_fs {
 	/* sb common to all (or all lower) layers */
 	struct super_block *same_lower_sb;
 	struct super_block *same_sb;
+	/* redirect by file handle */
+	bool redirect_fh;
 };
 
 enum ovl_path_type;
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index b8830ee..1b47557 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -17,6 +17,7 @@
 #include <linux/statfs.h>
 #include <linux/seq_file.h>
 #include <linux/posix_acl_xattr.h>
+#include <linux/exportfs.h>
 #include "overlayfs.h"
 #include "ovl_entry.h"
 
@@ -929,6 +930,19 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	else if (ufs->upper_mnt->mnt_sb == ufs->same_lower_sb)
 		ufs->same_sb = ufs->same_lower_sb;
 
+	/*
+	 * Redirect by file handle is used to find a lower entry in one of the
+	 * lower layers,  so the handle must be unique across all lower layers.
+	 * Therefore, enable redirect by file handle, only if all lower layers
+	 * are on the same sb which supports lookup by file handles.
+	 *
+	 * TODO: add support for looking up by (uuid,fh) tuple to enable
+	 *       redirect_fh for !same_lower_sb
+	 */
+	if (ufs->same_lower_sb && ufs->same_lower_sb->s_export_op &&
+	    ufs->same_lower_sb->s_export_op->fh_to_dentry)
+		ufs->redirect_fh = true;
+
 	if (remote)
 		sb->s_d_op = &ovl_reval_dentry_operations;
 	else
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index e015bc3..84f0c1f 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -240,6 +240,20 @@ void ovl_dentry_set_redirect(struct dentry *dentry, const char *redirect)
 	oe->redirect = redirect;
 }
 
+bool ovl_redirect_fh(struct super_block *sb)
+{
+	struct ovl_fs *ofs = sb->s_fs_info;
+
+	return ofs->redirect_fh;
+}
+
+void ovl_clear_redirect_fh(struct super_block *sb)
+{
+	struct ovl_fs *ofs = sb->s_fs_info;
+
+	ofs->redirect_fh = false;
+}
+
 void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry)
 {
 	struct ovl_entry *oe = dentry->d_fsdata;
-- 
2.7.4

Re: [PATCH v3 04/16] ovl: store file handle of lower inode on copy up

[Miklos Szeredi]

On Wed, Apr 26, 2017 at 11:35 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> Sometimes it is interesting to know if an upper file is pure
> upper or a copy up target, and if it is a copy up target, it
> may be interesting to find the copy up origin.
>
> This will be used to preserve lower inode numbers across copy up.
>
> Store the lower inode file handle in upper inode extended attribute
> overlay.origin.fh on copy up to use it later for these cases.
> Store the lower layer root file handle and lower filesystem uuid in
> overlay.origin.root and overlay.origin.uuid, to validate that we
> are looking for the origin file in the original layer.
>
> On failure to encode lower file handle, store an invalid 'null'
> handle, so we can always use the overlay.origin.fh xattr to tell
> between a copy up and a pure upper inode.
>
> If lower fs does not support NFS export ops or if not all lower
> layers are on the same fs, don't try to encode a lower file handle
> and use the 'null' handle instead.
>
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> ---
>  fs/overlayfs/copy_up.c   | 142 +++++++++++++++++++++++++++++++++++++++++++++++
>  fs/overlayfs/overlayfs.h |  29 ++++++++++
>  fs/overlayfs/ovl_entry.h |   2 +
>  fs/overlayfs/super.c     |  14 +++++
>  fs/overlayfs/util.c      |  14 +++++
>  5 files changed, 201 insertions(+)
>
> diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
> index 906ea6c..7cc7aea 100644
> --- a/fs/overlayfs/copy_up.c
> +++ b/fs/overlayfs/copy_up.c
> @@ -20,6 +20,8 @@
>  #include <linux/namei.h>
>  #include <linux/fdtable.h>
>  #include <linux/ratelimit.h>
> +#include <linux/mount.h>
> +#include <linux/exportfs.h>
>  #include "overlayfs.h"
>  #include "ovl_entry.h"
>
> @@ -232,6 +234,138 @@ int ovl_set_attr(struct dentry *upperdentry, struct kstat *stat)
>         return err;
>  }
>
> +static bool ovl_can_decode_fh(struct super_block *sb)
> +{
> +       return sb->s_export_op && sb->s_export_op->fh_to_dentry;
> +}
> +
> +static struct ovl_fh *ovl_decode_fh(struct dentry *lower)
> +{
> +       struct ovl_fh *fh;
> +       int fh_type, fh_len, dwords;
> +       void *buf = NULL;
> +       void *ret = NULL;
> +       int buflen = MAX_HANDLE_SZ;
> +       int err;
> +
> +       err = -EOPNOTSUPP;
> +       /* Do not encode file handle if we cannot decode it later */
> +       if (!ovl_can_decode_fh(lower->d_sb))
> +               goto out_err;
> +
> +       err = -ENOMEM;
> +       buf = kmalloc(buflen, GFP_TEMPORARY);
> +       if (!buf)
> +               goto out_err;
> +
> +       fh = buf;
> +       dwords = (buflen - offsetof(struct ovl_fh, fid)) >> 2;
> +       fh_type = exportfs_encode_fh(lower,
> +                                    (struct fid *)fh->fid,
> +                                    &dwords, 1);
> +       fh_len = (dwords << 2) + offsetof(struct ovl_fh, fid);
> +
> +       err = -EOVERFLOW;
> +       if (fh_len > buflen || fh_type <= 0 || fh_type == FILEID_INVALID)
> +               goto out_err;
> +
> +       fh->version = OVL_FH_VERSION;
> +       fh->magic = OVL_FH_MAGIC;
> +       fh->type = fh_type;
> +       fh->len = fh_len;
> +
> +       err = -ENOMEM;
> +       ret = kmalloc(fh_len, GFP_KERNEL);
> +       if (!ret)
> +               goto out_err;
> +
> +       memcpy(ret, buf, fh_len);
> +
> +       kfree(buf);
> +       return ret;
> +
> +out_err:
> +       pr_warn_ratelimited("overlay: failed to get redirect fh (%i)\n", err);
> +       kfree(buf);
> +       kfree(ret);
> +       return ERR_PTR(err);
> +}
> +
> +static const struct ovl_fh null_fh = {
> +       .version = OVL_FH_VERSION,
> +       .magic = OVL_FH_MAGIC,
> +       .type = FILEID_INVALID,
> +       .len = sizeof(struct ovl_fh),
> +};
> +
> +static int ovl_set_origin(struct dentry *dentry, struct dentry *upper)
> +{
> +       struct path lowerpath;
> +       struct super_block *lower_sb;
> +       const struct ovl_fh *fh = NULL;
> +       const struct ovl_fh *rootfh = NULL;
> +       int err;
> +
> +       ovl_path_lower(dentry, &lowerpath);
> +       if (WARN_ON(!lowerpath.mnt))
> +               return -EIO;
> +
> +       /*
> +        * Encoding a lower file handle where several layers are on the
> +        * same fs, require ecoding the layer root as well, because when
> +        * decoding the lower file handle we must provide the lowermnt.
> +        */
> +       lower_sb = lowerpath.mnt->mnt_sb;
> +       if (ovl_redirect_fh(dentry->d_sb) && ovl_can_decode_fh(lower_sb)) {
> +               fh = ovl_decode_fh(lowerpath.dentry);
> +               rootfh = ovl_decode_fh(lowerpath.mnt->mnt_root);
> +       }
> +       /*
> +        * On failure to encode lower fh, store an invalid 'null' fh, so
> +        * we can use the overlay.origin.fh xattr to distignuish between
> +        * a copy up and a pure upper inode.  If lower fs does not support
> +        * encoding fh, don't try to encode again (for any lower layer).
> +        */
> +       err = 0;
> +       if (IS_ERR_OR_NULL(fh)) {
> +               err = PTR_ERR(fh);
> +               fh = &null_fh;
> +       }
> +       if (IS_ERR_OR_NULL(rootfh)) {
> +               if (err != -EOPNOTSUPP)
> +                       err = PTR_ERR(rootfh);
> +               rootfh = NULL;
> +       }
> +       if (err == -EOPNOTSUPP) {
> +               pr_warn("overlay: file handle not supported by lower - turning off redirect_fh\n");
> +               ovl_clear_redirect_fh(dentry->d_sb);
> +       }
> +
> +       err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_FH, fh, fh->len, 0);
> +       if (err)
> +               goto out_err;
> +
> +       if (rootfh) {
> +               err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_ROOT, rootfh,
> +                                     rootfh->len, 0);
> +       }
> +       if (err)
> +               goto out_err;
> +
> +       if (fh != &null_fh) {
> +               err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_UUID,
> +                                     lower_sb->s_uuid,
> +                                     sizeof(lower_sb->s_uuid), 0);
> +       }
> +
> +out_err:
> +       if (fh != &null_fh)
> +               kfree(fh);
> +       return err;
> +       if (rootfh != &null_fh)
> +               kfree(rootfh);
> +}
> +
>  static int ovl_copy_up_locked(struct dentry *workdir, struct dentry *upperdir,
>                               struct dentry *dentry, struct path *lowerpath,
>                               struct kstat *stat, const char *link,
> @@ -316,6 +450,14 @@ static int ovl_copy_up_locked(struct dentry *workdir, struct dentry *upperdir,
>         if (err)
>                 goto out_cleanup;
>
> +       /*
> +        * Store identifier of lower inode in upper inode xattr to
> +        * allow lookup of the copy up origin inode.
> +        */
> +       err = ovl_set_origin(dentry, temp);
> +       if (err)
> +               goto out_cleanup;
> +
>         if (tmpfile)
>                 err = ovl_do_link(temp, udir, upper, true);
>         else
> diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
> index 48d0dae..2395dd7 100644
> --- a/fs/overlayfs/overlayfs.h
> +++ b/fs/overlayfs/overlayfs.h
> @@ -22,6 +22,33 @@ enum ovl_path_type {
>  #define OVL_XATTR_PREFIX XATTR_TRUSTED_PREFIX "overlay."
>  #define OVL_XATTR_OPAQUE OVL_XATTR_PREFIX "opaque"
>  #define OVL_XATTR_REDIRECT OVL_XATTR_PREFIX "redirect"
> +/*
> + * The tuple origin.{fh,layer,uuid} is a universal unique identifier
> + * for a copy up origin, where:
> + * origin.fh   - exported file handle of the lower file
> + * origin.root - exported file handle of the lower layer root
> + * origin.uuid - uuid of the lower filesystem
> + *
> + * origin.{fh,root} are stored in format of a variable length binary blob
> + * with struct ovl_fh header (total blob size up to 20 bytes).
> + * uuid is stored in raw format (16 bytes) as published by sb->s_uuid.
> + */
> +#define OVL_XATTR_ORIGIN_      OVL_XATTR_PREFIX "origin."
> +#define OVL_XATTR_ORIGIN_FH    OVL_XATTR_ORIGIN_ "fh"
> +#define OVL_XATTR_ORIGIN_ROOT  OVL_XATTR_ORIGIN_ "root"
> +#define OVL_XATTR_ORIGIN_UUID  OVL_XATTR_ORIGIN_ "uuid"


What do we gain by having these in separate xattrs?

They are binary blobs anyway, and fh is structured so it could
incorporate the others as well.   And "overlay.origin" would be a good
name for the combo.

> +
> +/* On-disk and in-memeory format for redirect by file handle */
> +#define OVL_FH_VERSION 0
> +#define OVL_FH_MAGIC   0xfb
> +
> +struct ovl_fh {
> +       unsigned char version;  /* 0 */
> +       unsigned char magic;    /* 0xfb */
> +       unsigned char len;      /* size of this header + size of fid */
> +       unsigned char type;     /* fid_type of fid */
> +       unsigned char fid[0];   /* file identifier */
> +} __packed;
>
>  #define OVL_ISUPPER_MASK 1UL
>
> @@ -175,6 +202,8 @@ bool ovl_redirect_dir(struct super_block *sb);
>  void ovl_clear_redirect_dir(struct super_block *sb);
>  const char *ovl_dentry_get_redirect(struct dentry *dentry);
>  void ovl_dentry_set_redirect(struct dentry *dentry, const char *redirect);
> +bool ovl_redirect_fh(struct super_block *sb);
> +void ovl_clear_redirect_fh(struct super_block *sb);
>  void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry);
>  void ovl_inode_init(struct inode *inode, struct inode *realinode,
>                     bool is_upper);
> diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
> index 41708bf..2172dc5 100644
> --- a/fs/overlayfs/ovl_entry.h
> +++ b/fs/overlayfs/ovl_entry.h
> @@ -32,6 +32,8 @@ struct ovl_fs {
>         /* sb common to all (or all lower) layers */
>         struct super_block *same_lower_sb;
>         struct super_block *same_sb;
> +       /* redirect by file handle */
> +       bool redirect_fh;
>  };
>
>  enum ovl_path_type;
> diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
> index b8830ee..1b47557 100644
> --- a/fs/overlayfs/super.c
> +++ b/fs/overlayfs/super.c
> @@ -17,6 +17,7 @@
>  #include <linux/statfs.h>
>  #include <linux/seq_file.h>
>  #include <linux/posix_acl_xattr.h>
> +#include <linux/exportfs.h>
>  #include "overlayfs.h"
>  #include "ovl_entry.h"
>
> @@ -929,6 +930,19 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
>         else if (ufs->upper_mnt->mnt_sb == ufs->same_lower_sb)
>                 ufs->same_sb = ufs->same_lower_sb;
>
> +       /*
> +        * Redirect by file handle is used to find a lower entry in one of the
> +        * lower layers,  so the handle must be unique across all lower layers.
> +        * Therefore, enable redirect by file handle, only if all lower layers
> +        * are on the same sb which supports lookup by file handles.
> +        *
> +        * TODO: add support for looking up by (uuid,fh) tuple to enable
> +        *       redirect_fh for !same_lower_sb
> +        */
> +       if (ufs->same_lower_sb && ufs->same_lower_sb->s_export_op &&
> +           ufs->same_lower_sb->s_export_op->fh_to_dentry)
> +               ufs->redirect_fh = true;
> +
>         if (remote)
>                 sb->s_d_op = &ovl_reval_dentry_operations;
>         else
> diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
> index e015bc3..84f0c1f 100644
> --- a/fs/overlayfs/util.c
> +++ b/fs/overlayfs/util.c
> @@ -240,6 +240,20 @@ void ovl_dentry_set_redirect(struct dentry *dentry, const char *redirect)
>         oe->redirect = redirect;
>  }
>
> +bool ovl_redirect_fh(struct super_block *sb)
> +{
> +       struct ovl_fs *ofs = sb->s_fs_info;
> +
> +       return ofs->redirect_fh;
> +}
> +
> +void ovl_clear_redirect_fh(struct super_block *sb)
> +{
> +       struct ovl_fs *ofs = sb->s_fs_info;
> +
> +       ofs->redirect_fh = false;
> +}
> +
>  void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry)
>  {
>         struct ovl_entry *oe = dentry->d_fsdata;
> --
> 2.7.4
>

Re: [PATCH v3 04/16] ovl: store file handle of lower inode on copy up

[Amir Goldstein]

On Thu, Apr 27, 2017 at 10:23 AM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> On Wed, Apr 26, 2017 at 11:35 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> Sometimes it is interesting to know if an upper file is pure
>> upper or a copy up target, and if it is a copy up target, it
>> may be interesting to find the copy up origin.
>>
>> This will be used to preserve lower inode numbers across copy up.
>>
>> Store the lower inode file handle in upper inode extended attribute
>> overlay.origin.fh on copy up to use it later for these cases.
>> Store the lower layer root file handle and lower filesystem uuid in
>> overlay.origin.root and overlay.origin.uuid, to validate that we
>> are looking for the origin file in the original layer.
>>
>> On failure to encode lower file handle, store an invalid 'null'
>> handle, so we can always use the overlay.origin.fh xattr to tell
>> between a copy up and a pure upper inode.
>>
>> If lower fs does not support NFS export ops or if not all lower
>> layers are on the same fs, don't try to encode a lower file handle
>> and use the 'null' handle instead.
>>
>> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
>> ---
>>  fs/overlayfs/copy_up.c   | 142 +++++++++++++++++++++++++++++++++++++++++++++++
>>  fs/overlayfs/overlayfs.h |  29 ++++++++++
>>  fs/overlayfs/ovl_entry.h |   2 +
>>  fs/overlayfs/super.c     |  14 +++++
>>  fs/overlayfs/util.c      |  14 +++++
>>  5 files changed, 201 insertions(+)
>>
>> diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
>> index 906ea6c..7cc7aea 100644
>> --- a/fs/overlayfs/copy_up.c
>> +++ b/fs/overlayfs/copy_up.c
>> @@ -20,6 +20,8 @@
>>  #include <linux/namei.h>
>>  #include <linux/fdtable.h>
>>  #include <linux/ratelimit.h>
>> +#include <linux/mount.h>
>> +#include <linux/exportfs.h>
>>  #include "overlayfs.h"
>>  #include "ovl_entry.h"
>>
>> @@ -232,6 +234,138 @@ int ovl_set_attr(struct dentry *upperdentry, struct kstat *stat)
>>         return err;
>>  }
>>
>> +static bool ovl_can_decode_fh(struct super_block *sb)
>> +{
>> +       return sb->s_export_op && sb->s_export_op->fh_to_dentry;
>> +}
>> +
>> +static struct ovl_fh *ovl_decode_fh(struct dentry *lower)
>> +{
>> +       struct ovl_fh *fh;
>> +       int fh_type, fh_len, dwords;
>> +       void *buf = NULL;
>> +       void *ret = NULL;
>> +       int buflen = MAX_HANDLE_SZ;
>> +       int err;
>> +
>> +       err = -EOPNOTSUPP;
>> +       /* Do not encode file handle if we cannot decode it later */
>> +       if (!ovl_can_decode_fh(lower->d_sb))
>> +               goto out_err;
>> +
>> +       err = -ENOMEM;
>> +       buf = kmalloc(buflen, GFP_TEMPORARY);
>> +       if (!buf)
>> +               goto out_err;
>> +
>> +       fh = buf;
>> +       dwords = (buflen - offsetof(struct ovl_fh, fid)) >> 2;
>> +       fh_type = exportfs_encode_fh(lower,
>> +                                    (struct fid *)fh->fid,
>> +                                    &dwords, 1);
>> +       fh_len = (dwords << 2) + offsetof(struct ovl_fh, fid);
>> +
>> +       err = -EOVERFLOW;
>> +       if (fh_len > buflen || fh_type <= 0 || fh_type == FILEID_INVALID)
>> +               goto out_err;
>> +
>> +       fh->version = OVL_FH_VERSION;
>> +       fh->magic = OVL_FH_MAGIC;
>> +       fh->type = fh_type;
>> +       fh->len = fh_len;
>> +
>> +       err = -ENOMEM;
>> +       ret = kmalloc(fh_len, GFP_KERNEL);
>> +       if (!ret)
>> +               goto out_err;
>> +
>> +       memcpy(ret, buf, fh_len);
>> +
>> +       kfree(buf);
>> +       return ret;
>> +
>> +out_err:
>> +       pr_warn_ratelimited("overlay: failed to get redirect fh (%i)\n", err);
>> +       kfree(buf);
>> +       kfree(ret);
>> +       return ERR_PTR(err);
>> +}
>> +
>> +static const struct ovl_fh null_fh = {
>> +       .version = OVL_FH_VERSION,
>> +       .magic = OVL_FH_MAGIC,
>> +       .type = FILEID_INVALID,
>> +       .len = sizeof(struct ovl_fh),
>> +};
>> +
>> +static int ovl_set_origin(struct dentry *dentry, struct dentry *upper)
>> +{
>> +       struct path lowerpath;
>> +       struct super_block *lower_sb;
>> +       const struct ovl_fh *fh = NULL;
>> +       const struct ovl_fh *rootfh = NULL;
>> +       int err;
>> +
>> +       ovl_path_lower(dentry, &lowerpath);
>> +       if (WARN_ON(!lowerpath.mnt))
>> +               return -EIO;
>> +
>> +       /*
>> +        * Encoding a lower file handle where several layers are on the
>> +        * same fs, require ecoding the layer root as well, because when
>> +        * decoding the lower file handle we must provide the lowermnt.
>> +        */
>> +       lower_sb = lowerpath.mnt->mnt_sb;
>> +       if (ovl_redirect_fh(dentry->d_sb) && ovl_can_decode_fh(lower_sb)) {
>> +               fh = ovl_decode_fh(lowerpath.dentry);
>> +               rootfh = ovl_decode_fh(lowerpath.mnt->mnt_root);
>> +       }
>> +       /*
>> +        * On failure to encode lower fh, store an invalid 'null' fh, so
>> +        * we can use the overlay.origin.fh xattr to distignuish between
>> +        * a copy up and a pure upper inode.  If lower fs does not support
>> +        * encoding fh, don't try to encode again (for any lower layer).
>> +        */
>> +       err = 0;
>> +       if (IS_ERR_OR_NULL(fh)) {
>> +               err = PTR_ERR(fh);
>> +               fh = &null_fh;
>> +       }
>> +       if (IS_ERR_OR_NULL(rootfh)) {
>> +               if (err != -EOPNOTSUPP)
>> +                       err = PTR_ERR(rootfh);
>> +               rootfh = NULL;
>> +       }
>> +       if (err == -EOPNOTSUPP) {
>> +               pr_warn("overlay: file handle not supported by lower - turning off redirect_fh\n");
>> +               ovl_clear_redirect_fh(dentry->d_sb);
>> +       }
>> +
>> +       err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_FH, fh, fh->len, 0);
>> +       if (err)
>> +               goto out_err;
>> +
>> +       if (rootfh) {
>> +               err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_ROOT, rootfh,
>> +                                     rootfh->len, 0);
>> +       }
>> +       if (err)
>> +               goto out_err;
>> +
>> +       if (fh != &null_fh) {
>> +               err = ovl_do_setxattr(upper, OVL_XATTR_ORIGIN_UUID,
>> +                                     lower_sb->s_uuid,
>> +                                     sizeof(lower_sb->s_uuid), 0);
>> +       }
>> +
>> +out_err:
>> +       if (fh != &null_fh)
>> +               kfree(fh);
>> +       return err;
>> +       if (rootfh != &null_fh)
>> +               kfree(rootfh);
>> +}
>> +
>>  static int ovl_copy_up_locked(struct dentry *workdir, struct dentry *upperdir,
>>                               struct dentry *dentry, struct path *lowerpath,
>>                               struct kstat *stat, const char *link,
>> @@ -316,6 +450,14 @@ static int ovl_copy_up_locked(struct dentry *workdir, struct dentry *upperdir,
>>         if (err)
>>                 goto out_cleanup;
>>
>> +       /*
>> +        * Store identifier of lower inode in upper inode xattr to
>> +        * allow lookup of the copy up origin inode.
>> +        */
>> +       err = ovl_set_origin(dentry, temp);
>> +       if (err)
>> +               goto out_cleanup;
>> +
>>         if (tmpfile)
>>                 err = ovl_do_link(temp, udir, upper, true);
>>         else
>> diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
>> index 48d0dae..2395dd7 100644
>> --- a/fs/overlayfs/overlayfs.h
>> +++ b/fs/overlayfs/overlayfs.h
>> @@ -22,6 +22,33 @@ enum ovl_path_type {
>>  #define OVL_XATTR_PREFIX XATTR_TRUSTED_PREFIX "overlay."
>>  #define OVL_XATTR_OPAQUE OVL_XATTR_PREFIX "opaque"
>>  #define OVL_XATTR_REDIRECT OVL_XATTR_PREFIX "redirect"
>> +/*
>> + * The tuple origin.{fh,layer,uuid} is a universal unique identifier
>> + * for a copy up origin, where:
>> + * origin.fh   - exported file handle of the lower file
>> + * origin.root - exported file handle of the lower layer root
>> + * origin.uuid - uuid of the lower filesystem
>> + *
>> + * origin.{fh,root} are stored in format of a variable length binary blob
>> + * with struct ovl_fh header (total blob size up to 20 bytes).
>> + * uuid is stored in raw format (16 bytes) as published by sb->s_uuid.
>> + */
>> +#define OVL_XATTR_ORIGIN_      OVL_XATTR_PREFIX "origin."
>> +#define OVL_XATTR_ORIGIN_FH    OVL_XATTR_ORIGIN_ "fh"
>> +#define OVL_XATTR_ORIGIN_ROOT  OVL_XATTR_ORIGIN_ "root"
>> +#define OVL_XATTR_ORIGIN_UUID  OVL_XATTR_ORIGIN_ "uuid"
>
>
> What do we gain by having these in separate xattrs?
>
> They are binary blobs anyway, and fh is structured so it could
> incorporate the others as well.   And "overlay.origin" would be a good
> name for the combo.
>

I agree. I felt more elegant this way, especially since origin.root
is optional, but it can also be optional in a binary blob.
I'll change it for v4.

Amir.

[PATCH v3 05/16] ovl: use an auxiliary var for overlay root entry

[Amir Goldstein]

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index d660177..17f9372 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -220,6 +220,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	const struct cred *old_cred;
 	struct ovl_fs *ofs = dentry->d_sb->s_fs_info;
 	struct ovl_entry *poe = dentry->d_parent->d_fsdata;
+	struct ovl_entry *roe = dentry->d_sb->s_root->d_fsdata;
 	struct path *stack = NULL;
 	struct dentry *upperdir, *upperdentry = NULL;
 	unsigned int ctr = 0;
@@ -259,7 +260,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 			if (!upperredirect)
 				goto out_put_upper;
 			if (d.redirect[0] == '/')
-				poe = dentry->d_sb->s_root->d_fsdata;
+				poe = roe;
 		}
 		if (d.opaque)
 			type |= __OVL_PATH_OPAQUE;
@@ -291,10 +292,8 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 		if (d.stop)
 			break;
 
-		if (d.redirect &&
-		    d.redirect[0] == '/' &&
-		    poe != dentry->d_sb->s_root->d_fsdata) {
-			poe = dentry->d_sb->s_root->d_fsdata;
+		if (d.redirect && d.redirect[0] == '/' && poe != roe) {
+			poe = roe;
 
 			/* Find the current layer on the root dentry */
 			for (i = 0; i < poe->numlower; i++)
-- 
2.7.4

[PATCH v3 06/16] ovl: factor out ovl_lookup_data()

[Amir Goldstein]

Split helper ovl_lookup_data() out of ovl_lookup_single().
The helper takes care of updating the ovl_lookup_data context
according to the dentry that was found in layer.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c | 36 +++++++++++++++++++++++-------------
 1 file changed, 23 insertions(+), 13 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 17f9372..b965785 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -96,22 +96,13 @@ static bool ovl_is_opaquedir(struct dentry *dentry)
 	return false;
 }
 
-static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
-			     const char *name, unsigned int namelen,
-			     size_t prelen, const char *post,
-			     struct dentry **ret)
+/* Update ovl_lookup_data struct from dentry found in layer */
+static int ovl_lookup_data(struct dentry *this, struct ovl_lookup_data *d,
+			   size_t prelen, const char *post,
+			   struct dentry **ret)
 {
-	struct dentry *this;
 	int err;
 
-	this = lookup_one_len_unlocked(name, base, namelen);
-	if (IS_ERR(this)) {
-		err = PTR_ERR(this);
-		this = NULL;
-		if (err == -ENOENT || err == -ENAMETOOLONG)
-			goto out;
-		goto out_err;
-	}
 	if (!this->d_inode)
 		goto put_and_out;
 
@@ -152,6 +143,25 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
 	return err;
 }
 
+static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
+			     const char *name, unsigned int namelen,
+			     size_t prelen, const char *post,
+			     struct dentry **ret)
+{
+	struct dentry *this = lookup_one_len_unlocked(name, base, namelen);
+	int err;
+
+	if (IS_ERR(this)) {
+		err = PTR_ERR(this);
+		*ret = NULL;
+		if (err == -ENOENT || err == -ENAMETOOLONG)
+			return 0;
+		return err;
+	}
+
+	return ovl_lookup_data(this, d, prelen, post, ret);
+}
+
 static int ovl_lookup_layer(struct dentry *base, struct ovl_lookup_data *d,
 			    struct dentry **ret)
 {
-- 
2.7.4

[PATCH v3 07/16] ovl: lookup redirect by file handle

[Amir Goldstein]

When overlay.origin.fh xattr is found in an upper inode, instead of lookup
of the dentry in next lower layer by name, first try to get it by calling
exportfs_decode_fh().

On failure to lookup by file handle to lower layer, fall back to lookup
by name with or without path redirect.

For now we only support following by file handle from upper if there is a
single lower layer, because fallback from lookup by file hande to lookup
by path in mid layers is not yet implemented.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c     | 158 +++++++++++++++++++++++++++++++++++++++++++++--
 fs/overlayfs/overlayfs.h |   1 +
 fs/overlayfs/util.c      |  14 +++++
 3 files changed, 168 insertions(+), 5 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index b965785..28d54e3 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -12,6 +12,8 @@
 #include <linux/namei.h>
 #include <linux/xattr.h>
 #include <linux/ratelimit.h>
+#include <linux/mount.h>
+#include <linux/exportfs.h>
 #include "overlayfs.h"
 #include "ovl_entry.h"
 
@@ -21,7 +23,10 @@ struct ovl_lookup_data {
 	bool opaque;
 	bool stop;
 	bool last;
-	char *redirect;
+	bool by_path;		/* redirect by path: */
+	char *redirect;		/* - path to follow */
+	bool by_fh;		/* redirect by file handle: */
+	struct ovl_fh *fh;	/* - file handle to follow */
 };
 
 static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
@@ -81,6 +86,52 @@ static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
 	goto err_free;
 }
 
+static struct ovl_fh *ovl_get_fh(struct dentry *dentry, const char *name)
+{
+	int res;
+	void *buf = NULL;
+
+	res = vfs_getxattr(dentry, name, NULL, 0);
+	if (res <= 0) {
+		if (res == -ENODATA || res == -EOPNOTSUPP)
+			return 0;
+		goto fail;
+	}
+	buf = kzalloc(res, GFP_TEMPORARY);
+	if (!buf) {
+		res = -ENOMEM;
+		goto fail;
+	}
+
+	res = vfs_getxattr(dentry, name, buf, res);
+	if (res < 0 || !ovl_redirect_fh_ok(buf, res))
+		goto fail;
+
+	return (struct ovl_fh *)buf;
+
+err_free:
+	kfree(buf);
+	return NULL;
+fail:
+	pr_warn_ratelimited("overlayfs: failed to get %s (%i)\n",
+			    name, res);
+	goto err_free;
+}
+
+static int ovl_check_redirect_fh(struct dentry *dentry,
+				 struct ovl_lookup_data *d)
+{
+	kfree(d->fh);
+	d->fh = ovl_get_fh(dentry, OVL_XATTR_ORIGIN_FH);
+	return 0;
+}
+
+static void ovl_reset_redirect_fh(struct ovl_lookup_data *d)
+{
+	kfree(d->fh);
+	d->fh = NULL;
+}
+
 static bool ovl_is_opaquedir(struct dentry *dentry)
 {
 	int res;
@@ -126,9 +177,16 @@ static int ovl_lookup_data(struct dentry *this, struct ovl_lookup_data *d,
 		d->stop = d->opaque = true;
 		goto out;
 	}
-	err = ovl_check_redirect(this, d, prelen, post);
-	if (err)
-		goto out_err;
+	if (d->by_path) {
+		err = ovl_check_redirect(this, d, prelen, post);
+		if (err)
+			goto out_err;
+	}
+	if (d->by_fh) {
+		err = ovl_check_redirect_fh(this, d);
+		if (err)
+			goto out_err;
+	}
 out:
 	*ret = this;
 	return 0;
@@ -202,6 +260,55 @@ static int ovl_lookup_layer(struct dentry *base, struct ovl_lookup_data *d,
 	return 0;
 }
 
+static struct dentry *ovl_decode_fh(struct vfsmount *mnt,
+				    const struct ovl_fh *fh,
+				    int (*acceptable)(void *, struct dentry *))
+{
+	int bytes = (fh->len - offsetof(struct ovl_fh, fid));
+
+	/*
+	 * When redirect_fh is disabled, 'invalid' file handles are stored
+	 * to indicate that this entry has been copied up.
+	 */
+	if (!bytes || (int)fh->type == FILEID_INVALID)
+		return ERR_PTR(-ESTALE);
+
+	/*
+	 * Several layers can be on the same fs and decoded dentry may be in
+	 * either one of those layers. We are looking for a match of dentry
+	 * and mnt to find out to which layer the decoded dentry belongs to.
+	 */
+	return exportfs_decode_fh(mnt, (struct fid *)fh->fid,
+				  bytes >> 2, (int)fh->type,
+				  acceptable, mnt);
+}
+
+/* Check if dentry belongs to this layer */
+static int ovl_dentry_in_layer(void *mnt, struct dentry *dentry)
+{
+	return is_subdir(dentry, ((struct vfsmount *)mnt)->mnt_root);
+}
+
+/* Lookup by file handle in a lower layer mounted at @mnt */
+static int ovl_lookup_layer_fh(struct vfsmount *mnt, struct ovl_lookup_data *d,
+			       struct dentry **ret)
+{
+	struct dentry *this = ovl_decode_fh(mnt, d->fh, ovl_dentry_in_layer);
+	int err;
+
+	if (IS_ERR(this)) {
+		err = PTR_ERR(this);
+		*ret = NULL;
+		if (err == -ESTALE)
+			return 0;
+		return err;
+	}
+
+	/* If found by file handle - don't follow that handle again */
+	ovl_reset_redirect_fh(d);
+	return ovl_lookup_data(this, d, 0, "", ret);
+}
+
 /*
  * Returns next layer in stack starting from top.
  * Returns -1 if this is the last layer.
@@ -246,7 +353,10 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 		.opaque = false,
 		.stop = false,
 		.last = !poe->numlower,
+		.by_path = true,
 		.redirect = NULL,
+		.by_fh = true,
+		.fh = NULL,
 	};
 
 	if (dentry->d_name.len > ofs->namelen)
@@ -276,7 +386,15 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 			type |= __OVL_PATH_OPAQUE;
 	}
 
-	if (!d.stop && poe->numlower) {
+	/*
+	 * For now we only support lower by fh in single layer, because
+	 * fallback from lookup by fh to lookup by path in mid layers for
+	 * merge directory is not yet implemented.
+	 */
+	if (!ofs->redirect_fh || ofs->numlower > 1)
+		ovl_reset_redirect_fh(&d);
+
+	if (!d.stop && (poe->numlower || d.fh)) {
 		err = -ENOMEM;
 		stack = kcalloc(ofs->numlower, sizeof(struct path),
 				GFP_TEMPORARY);
@@ -284,6 +402,35 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 			goto out_put_upper;
 	}
 
+	/* Try to lookup lower layers by file handle */
+	d.by_path = false;
+	if (!d.stop && d.fh) {
+		struct vfsmount *lowermnt = roe->lowerstack[0].mnt;
+
+		d.last = true;
+		err = ovl_lookup_layer_fh(lowermnt, &d, &this);
+		if (err)
+			goto out_put;
+
+		if (!this)
+			goto lookup_by_path;
+
+		stack[ctr].dentry = this;
+		stack[ctr].mnt = lowermnt;
+		ctr++;
+		/*
+		 * Found by fh - won't lookup by path.
+		 * TODO: set d.redirect to dentry_path(this), so
+		 *       lookup can continue by path in next layers
+		 */
+		d.stop = true;
+	}
+
+lookup_by_path:
+	/* Fallback to lookup lower layers by path */
+	d.by_path = true;
+	d.by_fh = false;
+	ovl_reset_redirect_fh(&d);
 	for (i = 0; !d.stop && i < poe->numlower; i++) {
 		struct path lowerpath = poe->lowerstack[i];
 
@@ -363,6 +510,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	dput(upperdentry);
 	kfree(upperredirect);
 out:
+	ovl_reset_redirect_fh(&d);
 	kfree(d.redirect);
 	revert_creds(old_cred);
 	return ERR_PTR(err);
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 2395dd7..ea23bef 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -204,6 +204,7 @@ const char *ovl_dentry_get_redirect(struct dentry *dentry);
 void ovl_dentry_set_redirect(struct dentry *dentry, const char *redirect);
 bool ovl_redirect_fh(struct super_block *sb);
 void ovl_clear_redirect_fh(struct super_block *sb);
+bool ovl_redirect_fh_ok(const char *redirect, size_t size);
 void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry);
 void ovl_inode_init(struct inode *inode, struct inode *realinode,
 		    bool is_upper);
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 84f0c1f..a36ca7b 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -254,6 +254,20 @@ void ovl_clear_redirect_fh(struct super_block *sb)
 	ofs->redirect_fh = false;
 }
 
+bool ovl_redirect_fh_ok(const char *redirect, size_t size)
+{
+	struct ovl_fh *fh = (void *)redirect;
+
+	if (size < sizeof(struct ovl_fh) || size < fh->len)
+		return false;
+
+	if (fh->version > OVL_FH_VERSION ||
+	    fh->magic != OVL_FH_MAGIC)
+		return false;
+
+	return true;
+}
+
 void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry)
 {
 	struct ovl_entry *oe = dentry->d_fsdata;
-- 
2.7.4

Re: [PATCH v3 07/16] ovl: lookup redirect by file handle

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

Hi Amir,

[auto build test ERROR on miklos-vfs/overlayfs-next]
[also build test ERROR on v4.11-rc8 next-20170427]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Amir-Goldstein/overlayfs-constant-inode-numbers/20170427-180531
base:   https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs-next
config: x86_64-rhel (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

>> ERROR: "is_subdir" [fs/overlayfs/overlay.ko] undefined!

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 38863 bytes --]

[PATCH v3 08/16] ovl: validate lower layer uuid and root on redirect by fh

[Amir Goldstein]

On copy up, we store xattr overlay.origin.uuid and
overlay.origin.root along with overlay.origin.fh.

Before decoding the file handle at overlay.origin.fh verify:
- All lower layers are on the same fs
- UUID of lower fs matches the stored uuid

If there are more than one lower layer, find the lower layer mount
in which origin.fh should be decoded by decoding origin.root and
matching the result to a lower layer root dentry.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 75 insertions(+), 1 deletion(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 28d54e3..970d8158 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -27,6 +27,8 @@ struct ovl_lookup_data {
 	char *redirect;		/* - path to follow */
 	bool by_fh;		/* redirect by file handle: */
 	struct ovl_fh *fh;	/* - file handle to follow */
+	struct ovl_fh *rootfh;	/* - file handle of layer root */
+	unsigned char uuid[16];	/* - uuid of layer filesystem */
 };
 
 static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
@@ -121,8 +123,24 @@ static struct ovl_fh *ovl_get_fh(struct dentry *dentry, const char *name)
 static int ovl_check_redirect_fh(struct dentry *dentry,
 				 struct ovl_lookup_data *d)
 {
+	int res;
+
 	kfree(d->fh);
 	d->fh = ovl_get_fh(dentry, OVL_XATTR_ORIGIN_FH);
+	kfree(d->rootfh);
+	d->rootfh = ovl_get_fh(dentry, OVL_XATTR_ORIGIN_ROOT);
+
+	res = vfs_getxattr(dentry, OVL_XATTR_ORIGIN_UUID, d->uuid,
+			   sizeof(d->uuid));
+	if (res == sizeof(d->uuid))
+		return 0;
+
+	if (res != -ENODATA && res != -EOPNOTSUPP) {
+		pr_warn_ratelimited("overlayfs: failed to get %s (%i)\n",
+				    OVL_XATTR_ORIGIN_UUID, res);
+	}
+
+	memset(d->uuid, 0, sizeof(d->uuid));
 	return 0;
 }
 
@@ -130,6 +148,9 @@ static void ovl_reset_redirect_fh(struct ovl_lookup_data *d)
 {
 	kfree(d->fh);
 	d->fh = NULL;
+	kfree(d->rootfh);
+	d->rootfh = NULL;
+	memset(d->uuid, 0, sizeof(d->uuid));
 }
 
 static bool ovl_is_opaquedir(struct dentry *dentry)
@@ -309,6 +330,53 @@ static int ovl_lookup_layer_fh(struct vfsmount *mnt, struct ovl_lookup_data *d,
 	return ovl_lookup_data(this, d, 0, "", ret);
 }
 
+static int ovl_is_dir(void *ctx, struct dentry *dentry)
+{
+	return d_is_dir(dentry);
+}
+
+/* Find lower layer index by layer root file handle and uuid */
+static int ovl_find_layer_by_fh(struct dentry *dentry, struct ovl_lookup_data *d)
+{
+	struct ovl_entry *roe = dentry->d_sb->s_root->d_fsdata;
+	struct super_block *lower_sb = ovl_same_lower_sb(dentry->d_sb);
+	struct dentry *this;
+	int i;
+
+	/*
+	 * For now, we only support lookup by fh for all lower layers on the
+	 * same sb.  Not all filesystems set sb->s_uuid.  For those who don't
+	 * this code will compare zeros, which at least ensures us that the
+	 * file handles are not crossing from filesystem with sb->s_uuid to
+	 * a filesystem without sb->s_uuid and vice versa.
+	 */
+	if (!lower_sb || memcmp(lower_sb->s_uuid, &d->uuid, sizeof(d->uuid)))
+		return -1;
+
+	/* Don't bother verifying rootfh with a single lower layer */
+	if (roe->numlower == 1)
+		return 0;
+
+	/*
+	 * Layer root dentries are pinned, there are no aliases for dirs, and
+	 * all lower layers are on the same sb.  If rootfh is correct,
+	 * exportfs_decode_fh() will find it in dcache and return the only
+	 * instance, regardless of the mnt argument and we can compare the
+	 * returned pointer with the pointers in lowerstack.
+	 */
+	this = ovl_decode_fh(roe->lowerstack[0].mnt, d->rootfh, ovl_is_dir);
+	if (IS_ERR(this))
+		return -1;
+
+	for (i = 0; i < roe->numlower; i++) {
+		if (this == roe->lowerstack[i].dentry)
+			break;
+	}
+
+	dput(this);
+	return i < roe->numlower ? i : -1;
+}
+
 /*
  * Returns next layer in stack starting from top.
  * Returns -1 if this is the last layer.
@@ -357,6 +425,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 		.redirect = NULL,
 		.by_fh = true,
 		.fh = NULL,
+		.rootfh = NULL,
 	};
 
 	if (dentry->d_name.len > ofs->namelen)
@@ -405,8 +474,13 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	/* Try to lookup lower layers by file handle */
 	d.by_path = false;
 	if (!d.stop && d.fh) {
-		struct vfsmount *lowermnt = roe->lowerstack[0].mnt;
+		struct vfsmount *lowermnt;
+		int layer = ovl_find_layer_by_fh(dentry, &d);
+
+		if (layer < 0 || layer >= roe->numlower)
+			goto lookup_by_path;
 
+		lowermnt = roe->lowerstack[layer].mnt;
 		d.last = true;
 		err = ovl_lookup_layer_fh(lowermnt, &d, &this);
 		if (err)
-- 
2.7.4

Re: [PATCH v3 08/16] ovl: validate lower layer uuid and root on redirect by fh

[Amir Goldstein]

On Thu, Apr 27, 2017 at 12:35 AM, Amir Goldstein <amir73il@gmail.com> wrote:
> On copy up, we store xattr overlay.origin.uuid and
> overlay.origin.root along with overlay.origin.fh.
>
> Before decoding the file handle at overlay.origin.fh verify:
> - All lower layers are on the same fs
> - UUID of lower fs matches the stored uuid
>
> If there are more than one lower layer, find the lower layer mount
> in which origin.fh should be decoded by decoding origin.root and
> matching the result to a lower layer root dentry.
>
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> ---
>  fs/overlayfs/namei.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 75 insertions(+), 1 deletion(-)
>
> diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
> index 28d54e3..970d8158 100644
> --- a/fs/overlayfs/namei.c
> +++ b/fs/overlayfs/namei.c
> @@ -27,6 +27,8 @@ struct ovl_lookup_data {
>         char *redirect;         /* - path to follow */
>         bool by_fh;             /* redirect by file handle: */
>         struct ovl_fh *fh;      /* - file handle to follow */
> +       struct ovl_fh *rootfh;  /* - file handle of layer root */
> +       unsigned char uuid[16]; /* - uuid of layer filesystem */
>  };
>
>  static int ovl_check_redirect(struct dentry *dentry, struct ovl_lookup_data *d,
> @@ -121,8 +123,24 @@ static struct ovl_fh *ovl_get_fh(struct dentry *dentry, const char *name)
>  static int ovl_check_redirect_fh(struct dentry *dentry,
>                                  struct ovl_lookup_data *d)
>  {
> +       int res;
> +
>         kfree(d->fh);
>         d->fh = ovl_get_fh(dentry, OVL_XATTR_ORIGIN_FH);
> +       kfree(d->rootfh);
> +       d->rootfh = ovl_get_fh(dentry, OVL_XATTR_ORIGIN_ROOT);
> +
> +       res = vfs_getxattr(dentry, OVL_XATTR_ORIGIN_UUID, d->uuid,
> +                          sizeof(d->uuid));
> +       if (res == sizeof(d->uuid))
> +               return 0;
> +
> +       if (res != -ENODATA && res != -EOPNOTSUPP) {
> +               pr_warn_ratelimited("overlayfs: failed to get %s (%i)\n",
> +                                   OVL_XATTR_ORIGIN_UUID, res);
> +       }
> +
> +       memset(d->uuid, 0, sizeof(d->uuid));
>         return 0;
>  }
>
> @@ -130,6 +148,9 @@ static void ovl_reset_redirect_fh(struct ovl_lookup_data *d)
>  {
>         kfree(d->fh);
>         d->fh = NULL;
> +       kfree(d->rootfh);
> +       d->rootfh = NULL;
> +       memset(d->uuid, 0, sizeof(d->uuid));
>  }
>
>  static bool ovl_is_opaquedir(struct dentry *dentry)
> @@ -309,6 +330,53 @@ static int ovl_lookup_layer_fh(struct vfsmount *mnt, struct ovl_lookup_data *d,
>         return ovl_lookup_data(this, d, 0, "", ret);
>  }
>
> +static int ovl_is_dir(void *ctx, struct dentry *dentry)
> +{
> +       return d_is_dir(dentry);
> +}
> +
> +/* Find lower layer index by layer root file handle and uuid */
> +static int ovl_find_layer_by_fh(struct dentry *dentry, struct ovl_lookup_data *d)
> +{
> +       struct ovl_entry *roe = dentry->d_sb->s_root->d_fsdata;
> +       struct super_block *lower_sb = ovl_same_lower_sb(dentry->d_sb);
> +       struct dentry *this;
> +       int i;
> +
> +       /*
> +        * For now, we only support lookup by fh for all lower layers on the
> +        * same sb.  Not all filesystems set sb->s_uuid.  For those who don't
> +        * this code will compare zeros, which at least ensures us that the
> +        * file handles are not crossing from filesystem with sb->s_uuid to
> +        * a filesystem without sb->s_uuid and vice versa.

Nah. On second thought, it's better to disable redirect_fh at mount time
if lower s_uuid is NULL.

> +        */
> +       if (!lower_sb || memcmp(lower_sb->s_uuid, &d->uuid, sizeof(d->uuid)))
> +               return -1;
> +
> +       /* Don't bother verifying rootfh with a single lower layer */
> +       if (roe->numlower == 1)
> +               return 0;
> +
> +       /*
> +        * Layer root dentries are pinned, there are no aliases for dirs, and
> +        * all lower layers are on the same sb.  If rootfh is correct,
> +        * exportfs_decode_fh() will find it in dcache and return the only
> +        * instance, regardless of the mnt argument and we can compare the
> +        * returned pointer with the pointers in lowerstack.
> +        */
> +       this = ovl_decode_fh(roe->lowerstack[0].mnt, d->rootfh, ovl_is_dir);
> +       if (IS_ERR(this))
> +               return -1;
> +
> +       for (i = 0; i < roe->numlower; i++) {
> +               if (this == roe->lowerstack[i].dentry)
> +                       break;
> +       }
> +
> +       dput(this);
> +       return i < roe->numlower ? i : -1;
> +}
> +
>  /*
>   * Returns next layer in stack starting from top.
>   * Returns -1 if this is the last layer.
> @@ -357,6 +425,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
>                 .redirect = NULL,
>                 .by_fh = true,
>                 .fh = NULL,
> +               .rootfh = NULL,
>         };
>
>         if (dentry->d_name.len > ofs->namelen)
> @@ -405,8 +474,13 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
>         /* Try to lookup lower layers by file handle */
>         d.by_path = false;
>         if (!d.stop && d.fh) {
> -               struct vfsmount *lowermnt = roe->lowerstack[0].mnt;
> +               struct vfsmount *lowermnt;
> +               int layer = ovl_find_layer_by_fh(dentry, &d);
> +
> +               if (layer < 0 || layer >= roe->numlower)
> +                       goto lookup_by_path;
>
> +               lowermnt = roe->lowerstack[layer].mnt;
>                 d.last = true;
>                 err = ovl_lookup_layer_fh(lowermnt, &d, &this);
>                 if (err)
> --
> 2.7.4
>

[PATCH v3 09/16] ovl: lookup non-dir inode copy up origin

[Amir Goldstein]

When non directory upper has overlay.origin.fh xattr, lookup in lower
layers by file handle or by path to find the copy up origin inode.

Until this change, a non-dir dentry could have had oe->numlower == 1
with oe->lowerstack[0] pointing at the copy up origin path, right
after copy up, but not when a non-dir dentry was created by ovl_lookup().

After this change, a non-dir dentry could be pointing at the copy up
origin after ovl_lookup(), as long as the copy up was done by overlayfs
that had redirect_fh support.

Non-dir entries that were copied up by overlayfs without redirect_fh
support will look the same as pure upper non-dir entries.

This is going to be used for persistent inode numbers across copy up.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 970d8158..ad2c784 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -188,15 +188,16 @@ static int ovl_lookup_data(struct dentry *this, struct ovl_lookup_data *d,
 		goto put_and_out;
 	}
 	if (!d_can_lookup(this)) {
-		d->stop = true;
-		if (d->is_dir)
+		if (d->is_dir) {
+			d->stop = true;
 			goto put_and_out;
-		goto out;
-	}
-	d->is_dir = true;
-	if (!d->last && ovl_is_opaquedir(this)) {
-		d->stop = d->opaque = true;
-		goto out;
+		}
+	} else {
+		d->is_dir = true;
+		if (!d->last && ovl_is_opaquedir(this)) {
+			d->stop = d->opaque = true;
+			goto out;
+		}
 	}
 	if (d->by_path) {
 		err = ovl_check_redirect(this, d, prelen, post);
@@ -208,6 +209,9 @@ static int ovl_lookup_data(struct dentry *this, struct ovl_lookup_data *d,
 		if (err)
 			goto out_err;
 	}
+	/* No redirect for non-dir means pure upper */
+	if (!d->is_dir)
+		d->stop = !d->fh && !d->redirect;
 out:
 	*ret = this;
 	return 0;
@@ -456,11 +460,11 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	}
 
 	/*
-	 * For now we only support lower by fh in single layer, because
-	 * fallback from lookup by fh to lookup by path in mid layers for
-	 * merge directory is not yet implemented.
+	 * For now we only support lookup by fh in single layer for directory,
+	 * because fallback from lookup by fh to lookup by path in mid layers
+	 * for merge directory is not yet implemented.
 	 */
-	if (!ofs->redirect_fh || ofs->numlower > 1)
+	if (!ofs->redirect_fh || (d.is_dir && ofs->numlower > 1))
 		ovl_reset_redirect_fh(&d);
 
 	if (!d.stop && (poe->numlower || d.fh)) {
@@ -520,7 +524,8 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 		stack[ctr].mnt = lowerpath.mnt;
 		ctr++;
 
-		if (d.stop)
+		/* Do not follow non-dir copy up origin more than once */
+		if (d.stop || !d.is_dir)
 			break;
 
 		if (d.redirect && d.redirect[0] == '/' && poe != roe) {
-- 
2.7.4

[PATCH v3 10/16] ovl: set the COPYUP type flag for non-dirs

[Amir Goldstein]

For directory entries, non zero oe->numlower implies OVL_TYPE_MERGE.
Define a new type flag OVL_TYPE_COPYUP to indicate that an entry is
a target of a copy up.

For directory entries COPYUP = MERGE && UPPER. For non-dir entries
non zero oe->numlower implies COPYUP, but COPYUP does not imply
non zero oe->numlower.  COPYUP can also be set on lookup when detecting
an overlay.fh xattr on a non-dir, even if that fh cannot be followed.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c     |  3 +++
 fs/overlayfs/overlayfs.h |  2 ++
 fs/overlayfs/util.c      | 12 ++++++++----
 3 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index ad2c784..3ebde1b 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -457,6 +457,9 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 		}
 		if (d.opaque)
 			type |= __OVL_PATH_OPAQUE;
+		/* overlay.fh xattr implies this is a copy up */
+		if (d.fh)
+			type |= __OVL_PATH_COPYUP;
 	}
 
 	/*
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index ea23bef..d9633b3 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -13,11 +13,13 @@ enum ovl_path_type {
 	__OVL_PATH_UPPER	= (1 << 0),
 	__OVL_PATH_MERGE	= (1 << 1),
 	__OVL_PATH_OPAQUE	= (1 << 2),
+	__OVL_PATH_COPYUP	= (1 << 3),
 };
 
 #define OVL_TYPE_UPPER(type)	((type) & __OVL_PATH_UPPER)
 #define OVL_TYPE_MERGE(type)	((type) & __OVL_PATH_MERGE)
 #define OVL_TYPE_OPAQUE(type)	((type) & __OVL_PATH_OPAQUE)
+#define OVL_TYPE_COPYUP(type)	((type) & __OVL_PATH_COPYUP)
 
 #define OVL_XATTR_PREFIX XATTR_TRUSTED_PREFIX "overlay."
 #define OVL_XATTR_OPAQUE OVL_XATTR_PREFIX "opaque"
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index a36ca7b..931d199 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -101,11 +101,15 @@ enum ovl_path_type ovl_update_type(struct dentry *dentry, bool is_dir)
 	if (oe->__upperdentry) {
 		type |= __OVL_PATH_UPPER;
 		/*
-		 * Non-dir dentry can hold lower dentry from before
-		 * copy-up.
+		 * oe->numlower implies a copy up, but copy up does not imply
+		 * oe->numlower.  It can also be set on lookup when detecting
+		 * an overlay.fh xattr on a non-dir that cannot be followed.
 		 */
-		if (oe->numlower && is_dir)
-			type |= __OVL_PATH_MERGE;
+		if (oe->numlower) {
+			type |= __OVL_PATH_COPYUP;
+			if (is_dir)
+				type |= __OVL_PATH_MERGE;
+		}
 	} else {
 		if (oe->numlower > 1)
 			type |= __OVL_PATH_MERGE;
-- 
2.7.4

[PATCH v3 11/16] ovl: redirect non-dir by path on rename

[Amir Goldstein]

When a non-dir COPYUP type entry is being renamed, set its
overlay.redirect xattr, just the same as when renaming a lower
or merge directory.

This will be used to find the copy up original of non-dir inodes
in case the lower layers do not support lookup by file handle.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/dir.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index bfabc65..7372c15 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -795,6 +795,13 @@ static bool ovl_type_merge_or_lower(struct dentry *dentry)
 	return OVL_TYPE_MERGE(type) || !OVL_TYPE_UPPER(type);
 }
 
+static bool ovl_type_copyup(struct dentry *dentry)
+{
+	enum ovl_path_type type = ovl_path_type(dentry);
+
+	return OVL_TYPE_COPYUP(type);
+}
+
 static bool ovl_can_move(struct dentry *dentry)
 {
 	return ovl_redirect_dir(dentry->d_sb) ||
@@ -1022,6 +1029,8 @@ static int ovl_rename(struct inode *olddir, struct dentry *old,
 			err = ovl_set_opaque(old, olddentry);
 		if (err)
 			goto out_dput;
+	} else if (ovl_type_copyup(old)) {
+		err = ovl_set_redirect(old, samedir);
 	}
 	if (!overwrite && new_is_dir) {
 		if (ovl_type_merge_or_lower(new))
@@ -1030,6 +1039,8 @@ static int ovl_rename(struct inode *olddir, struct dentry *old,
 			err = ovl_set_opaque(new, newdentry);
 		if (err)
 			goto out_dput;
+	} else if (!overwrite && ovl_type_copyup(new)) {
+		err = ovl_set_redirect(new, samedir);
 	}
 
 	err = ovl_do_rename(old_upperdir->d_inode, olddentry,
-- 
2.7.4

[PATCH v3 12/16] ovl: constant st_ino/st_dev across copy up

[Amir Goldstein]

When getting attributes for overlay inode of path type COPYUP,
get the inode and dev numbers from the copy up origin inode.

This results in constant and persistent st_ino/st_dev representation
of files in overlay mount before and after copy up as well as after
mount cycle.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/inode.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 17b8418..3615a52 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -60,15 +60,25 @@ int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 static int ovl_getattr(const struct path *path, struct kstat *stat,
 		       u32 request_mask, unsigned int flags)
 {
-	struct dentry *dentry = path->dentry;
+	struct dentry *lower, *dentry = path->dentry;
 	struct path realpath;
 	const struct cred *old_cred;
+	enum ovl_path_type type;
 	int err;
 
-	ovl_path_real(dentry, &realpath);
+	type = ovl_path_real(dentry, &realpath);
 	old_cred = ovl_override_creds(dentry->d_sb);
 	err = vfs_getattr(&realpath, stat, request_mask, flags);
 	revert_creds(old_cred);
+	if (err)
+		return err;
+
+	lower = ovl_dentry_lower(dentry);
+	if (OVL_TYPE_COPYUP(type) && lower) {
+		stat->dev = lower->d_sb->s_dev;
+		stat->ino = lower->d_inode->i_ino;
+	}
+
 	return err;
 }
 
-- 
2.7.4

[PATCH v3 13/16] ovl: persistent inode number for directories

[Amir Goldstein]

stat(2) on overlay directories reports the overlay temp inode
number, which is constant across copy up, but is not persistent.

When all layers are on the same fs, report the upper most lower inode
(a.k.a stable inode) number for directories.

This inode number is persistent, unique across the overlay mount and
constant across copy up.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/dir.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index 7372c15..da65c02 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -154,8 +154,23 @@ static int ovl_dir_getattr(const struct path *path, struct kstat *stat,
 	if (err)
 		return err;
 
+	/*
+	 * Always use the overlay bdev for directories, so 'find -xdev' will
+	 * scan the entire overlay mount and won't cross the overlay mount
+	 * boundaries.
+	 */
 	stat->dev = dentry->d_sb->s_dev;
-	stat->ino = dentry->d_inode->i_ino;
+	/*
+	 * When all layers are not on the same fs, the pair real inode numbers
+	 * and overlay bdev is not unique, so use the non persistent overlay
+	 * inode number.
+	 * When all layers are on the same fs, use the stable inode number,
+	 * which is persistent, unique and constant across copy up.
+	 */
+	if (!ovl_same_sb(dentry->d_sb))
+		stat->ino = dentry->d_inode->i_ino;
+	else if (OVL_TYPE_UPPER(type) && OVL_TYPE_MERGE(type))
+		stat->ino = ovl_dentry_lower(dentry)->d_inode->i_ino;
 
 	/*
 	 * It's probably not worth it to count subdirs to get the
-- 
2.7.4

[PATCH v3 14/16] ovl: fix du --one-file-system on overlay mount

[Amir Goldstein]

Overlay directory inodes report overlay bdev to stat(2).
Overlay non-dir inodes report real bdev and real ino to stat(2).

Due to the different bdev values for dir and non-dir inodes, when executing
the command du -x on an overlay mount, the result is wrong because non-dirs
are not accounted for in the overlay bdev usage.

The reasons for this bdev inconsistecy is:
1. The overlay ino is not persistent, so real ino is used for non-dirs
2. The tupple overlay bdev and real ino is not unique, so real bdev is
   used for non-dirs

In case all overlay layers are on the same underlying fs, the tupple
from reason 2 above is unique, so use this tupple for non-dirs to get
the correct result from du -x.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/inode.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 3615a52..39c3bb0 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -78,6 +78,13 @@ static int ovl_getattr(const struct path *path, struct kstat *stat,
 		stat->dev = lower->d_sb->s_dev;
 		stat->ino = lower->d_inode->i_ino;
 	}
+	/*
+	 * When all layers are on same fs, the tupple overlay bdev
+	 * and real inode ino is unique, so it is preferred to expose
+	 * overlay bdev for overlay inodes for things like du -x.
+	 */
+	if (ovl_same_sb(dentry->d_sb))
+		stat->dev = dentry->d_sb->s_dev;
 
 	return err;
 }
-- 
2.7.4

[PATCH v3 15/16] ovl: persistent inode numbers for hardlinks

[Amir Goldstein]

An upper type non directory dentry that is a copy up target
should have a reference to its lower copy up origin.

There are three ways for an upper type dentry to be instantiated:
1. A lower type dentry that is being copied up
2. An entry that is found in upper dir by ovl_lookup()
3. A negative dentry is hardlinked to an upper type dentry

In the first case, the lower reference is set before copy up.
In the second case, the lower reference is found by ovl_lookup().
In the last case of hardlinked upper dentry, it is not easy to
update the lower reference of the negative dentry.  Instead,
drop the newly hardlinked negative dentry from dcache and let
the next access call ovl_lookup() to find its lower reference.

This makes sure that the inode number reported by stat(2) after
the hardlink is created is the same inode number that will be
reported by stat(2) after mount cycle, which is the inode number
of the lower copy up origin of the hardlink source.

NOTE that this does not fix breaking of lower hardlinks on copy
up, but it will result in stat(2) reporting the same inode number
for all the upper broken hardlinks.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/dir.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index da65c02..5d64550 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -197,6 +197,9 @@ static void ovl_instantiate(struct dentry *dentry, struct inode *inode,
 		inc_nlink(inode);
 	}
 	d_instantiate(dentry, inode);
+	/* Force lookup of new upper hardlink to find its lower */
+	if (hardlink)
+		d_drop(dentry);
 }
 
 static bool ovl_type_merge(struct dentry *dentry)
-- 
2.7.4

[PATCH v3 16/16] ovl: update documentation w.r.t. constant inode numbers

[Amir Goldstein]

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 Documentation/filesystems/overlayfs.txt | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/Documentation/filesystems/overlayfs.txt b/Documentation/filesystems/overlayfs.txt
index 634d03e..437e38d 100644
--- a/Documentation/filesystems/overlayfs.txt
+++ b/Documentation/filesystems/overlayfs.txt
@@ -21,11 +21,17 @@ from accessing the corresponding object from the original filesystem.
 This is most obvious from the 'st_dev' field returned by stat(2).
 
 While directories will report an st_dev from the overlay-filesystem,
-all non-directory objects will report an st_dev from the lower or
-upper filesystem that is providing the object.  Similarly st_ino will
-only be unique when combined with st_dev, and both of these can change
-over the lifetime of a non-directory object.  Many applications and
-tools ignore these values and will not be affected.
+non-directory objects may report an st_dev from the lower or upper
+filesystem that is providing the object.  Similarly st_ino will only
+be unique when combined with st_dev.  Many applications and tools
+ignore these values and will not be affected.
+
+In the special case of all overlay layers on the same underlying
+filesystem, all objects will report an st_dev from the overlay
+filesystem and st_ino from the underlying filesystem.  This will
+make the overlay mount more comliant with filesystem scanners and
+overlay objects will be distinguishable from the corresponding
+objects from the original filesystem.
 
 Upper and Lower
 ---------------
@@ -198,8 +204,7 @@ Non-standard behavior
 ---------------------
 
 The copy_up operation essentially creates a new, identical file and
-moves it over to the old name.  The new file may be on a different
-filesystem, so both st_dev and st_ino of the file may change.
+moves it over to the old name.
 
 Any open files referring to this inode will access the old data.
 
-- 
2.7.4