[PATCH 0/6] overlayfs + casefolding

[PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

This series allows overlayfs and casefolding to safely be used on the
same filesystem by providing exclusion to ensure that overlayfs never
has to deal with casefolded directories.

Currently, overlayfs can't be used _at all_ if a filesystem even
supports casefolding, which is really nasty for users.

Components:

- filesystem has to track, for each directory, "does any _descendent_
  have casefolding enabled"

- new inode flag to pass this to VFS layer

- new dcache methods for providing refs for overlayfs, and filesystem
  methods for safely clearing this flag

- new superblock flag for indicating to overlayfs & dcache "filesystem
  supports casefolding, it's safe to use provided new dcache methods are
  used"

Kent Overstreet (6):
  bcachefs: BCH_INODE_has_case_insensitive
  darray: lift from bcachefs
  fs: SB_CASEFOLD
  fs: dcache locking for exlusion between overlayfs, casefolding
  bcachefs: Hook up d_casefold_enable()
  overlayfs: Support casefolded filesystems

 MAINTAINERS                             |   7 +
 fs/bcachefs/Makefile                    |   1 -
 fs/bcachefs/bcachefs_format.h           |   3 +-
 fs/bcachefs/btree_node_scan_types.h     |   2 +-
 fs/bcachefs/btree_types.h               |   2 +-
 fs/bcachefs/btree_update.c              |   1 +
 fs/bcachefs/btree_write_buffer_types.h  |   2 +-
 fs/bcachefs/disk_accounting_types.h     |   2 +-
 fs/bcachefs/fs.c                        |  45 +++++-
 fs/bcachefs/fsck.c                      |  12 +-
 fs/bcachefs/inode.c                     |   8 +-
 fs/bcachefs/inode.h                     |   2 +-
 fs/bcachefs/inode_format.h              |   7 +-
 fs/bcachefs/journal_io.h                |   2 +-
 fs/bcachefs/journal_sb.c                |   2 +-
 fs/bcachefs/namei.c                     | 166 +++++++++++++++++++++-
 fs/bcachefs/namei.h                     |   5 +
 fs/bcachefs/rcu_pending.c               |   3 +-
 fs/bcachefs/sb-downgrade.c              |   9 +-
 fs/bcachefs/sb-errors_format.h          |   4 +-
 fs/bcachefs/sb-errors_types.h           |   2 +-
 fs/bcachefs/sb-members.h                |   3 +-
 fs/bcachefs/snapshot_types.h            |   3 +-
 fs/bcachefs/subvolume.h                 |   1 -
 fs/bcachefs/thread_with_file_types.h    |   2 +-
 fs/bcachefs/util.h                      |  28 +---
 fs/dcache.c                             | 177 ++++++++++++++++++++++++
 fs/libfs.c                              |   1 +
 fs/overlayfs/params.c                   |  20 ++-
 fs/overlayfs/util.c                     |  19 ++-
 {fs/bcachefs => include/linux}/darray.h |  70 +++++-----
 include/linux/darray_types.h            |  33 +++++
 include/linux/dcache.h                  |  10 ++
 include/linux/fs.h                      |   4 +
 lib/Makefile                            |   2 +-
 {fs/bcachefs => lib}/darray.c           |   9 +-
 36 files changed, 571 insertions(+), 98 deletions(-)
 rename {fs/bcachefs => include/linux}/darray.h (64%)
 create mode 100644 include/linux/darray_types.h
 rename {fs/bcachefs => lib}/darray.c (75%)

-- 
2.49.0

[PATCH 1/6] bcachefs: BCH_INODE_has_case_insensitive

[Kent Overstreet]

Add a flag for tracking whether a directory has case-insensitive
descendents - so that overlayfs can disallow mounting, even though the
filesystem supports case insensitivity.

This is a new on disk format version, with a (cheap) upgrade to ensure
the flag is correctly set on existing inodes.

Create, rename and fssetxattr are all plumbed to ensure the new flag is
set, and we've got new fsck code that hooks into check_inode(0.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
---
 fs/bcachefs/bcachefs_format.h  |   3 +-
 fs/bcachefs/fs.c               |   4 +
 fs/bcachefs/fsck.c             |  10 +-
 fs/bcachefs/inode.c            |   8 +-
 fs/bcachefs/inode.h            |   2 +-
 fs/bcachefs/inode_format.h     |   7 +-
 fs/bcachefs/namei.c            | 166 ++++++++++++++++++++++++++++++++-
 fs/bcachefs/namei.h            |   5 +
 fs/bcachefs/sb-downgrade.c     |   6 +-
 fs/bcachefs/sb-errors_format.h |   4 +-
 10 files changed, 202 insertions(+), 13 deletions(-)

diff --git a/fs/bcachefs/bcachefs_format.h b/fs/bcachefs/bcachefs_format.h
index 5900ff3715c6..b4a04df5ea95 100644
--- a/fs/bcachefs/bcachefs_format.h
+++ b/fs/bcachefs/bcachefs_format.h
@@ -699,7 +699,8 @@ struct bch_sb_field_ext {
 	x(casefolding,			BCH_VERSION(1, 24))		\
 	x(extent_flags,			BCH_VERSION(1, 25))		\
 	x(snapshot_deletion_v2,		BCH_VERSION(1, 26))		\
-	x(fast_device_removal,		BCH_VERSION(1, 27))
+	x(fast_device_removal,		BCH_VERSION(1, 27))		\
+	x(inode_has_case_insensitive,	BCH_VERSION(1, 28))
 
 enum bcachefs_metadata_version {
 	bcachefs_metadata_version_min = 9,
diff --git a/fs/bcachefs/fs.c b/fs/bcachefs/fs.c
index c863dcae5062..0b5d52895e05 100644
--- a/fs/bcachefs/fs.c
+++ b/fs/bcachefs/fs.c
@@ -1681,6 +1681,10 @@ static int fssetxattr_inode_update_fn(struct btree_trans *trans,
 		bi->bi_casefold = s->casefold + 1;
 		bi->bi_fields_set |= BIT(Inode_opt_casefold);
 
+		ret = bch2_maybe_propagate_has_case_insensitive(trans, inode_inum(inode), bi);
+		if (ret)
+			return ret;
+
 #else
 		printk(KERN_ERR "Cannot use casefolding on a kernel without CONFIG_UNICODE\n");
 		return -EOPNOTSUPP;
diff --git a/fs/bcachefs/fsck.c b/fs/bcachefs/fsck.c
index a0afb4c2a6f5..338309d0a570 100644
--- a/fs/bcachefs/fsck.c
+++ b/fs/bcachefs/fsck.c
@@ -264,7 +264,7 @@ static int lookup_lostfound(struct btree_trans *trans, u32 snapshot,
 	u64 cpu = raw_smp_processor_id();
 
 	bch2_inode_init_early(c, lostfound);
-	bch2_inode_init_late(lostfound, now, 0, 0, S_IFDIR|0700, 0, &root_inode);
+	bch2_inode_init_late(c, lostfound, now, 0, 0, S_IFDIR|0700, 0, &root_inode);
 	lostfound->bi_dir = root_inode.bi_inum;
 	lostfound->bi_snapshot = le32_to_cpu(st.root_snapshot);
 
@@ -543,7 +543,7 @@ static int reconstruct_subvol(struct btree_trans *trans, u32 snapshotid, u32 sub
 		u64 cpu = raw_smp_processor_id();
 
 		bch2_inode_init_early(c, &new_inode);
-		bch2_inode_init_late(&new_inode, bch2_current_time(c), 0, 0, S_IFDIR|0755, 0, NULL);
+		bch2_inode_init_late(c, &new_inode, bch2_current_time(c), 0, 0, S_IFDIR|0755, 0, NULL);
 
 		new_inode.bi_subvol = subvolid;
 
@@ -633,7 +633,7 @@ static int reconstruct_inode(struct btree_trans *trans, enum btree_id btree, u32
 
 	struct bch_inode_unpacked new_inode;
 	bch2_inode_init_early(c, &new_inode);
-	bch2_inode_init_late(&new_inode, bch2_current_time(c), 0, 0, i_mode|0600, 0, NULL);
+	bch2_inode_init_late(c, &new_inode, bch2_current_time(c), 0, 0, i_mode|0600, 0, NULL);
 	new_inode.bi_size = i_size;
 	new_inode.bi_inum = inum;
 	new_inode.bi_snapshot = snapshot;
@@ -1135,6 +1135,10 @@ static int check_inode(struct btree_trans *trans,
 			goto err;
 	}
 
+	ret = bch2_check_inode_has_case_insensitive(trans, &u, &s->ids, &do_update);
+	if (ret)
+		goto err;
+
 	if (u.bi_dir || u.bi_dir_offset) {
 		ret = check_inode_dirent_inode(trans, &u, &do_update);
 		if (ret)
diff --git a/fs/bcachefs/inode.c b/fs/bcachefs/inode.c
index 1d7c3e6b9089..dbcb95ad1e33 100644
--- a/fs/bcachefs/inode.c
+++ b/fs/bcachefs/inode.c
@@ -907,7 +907,8 @@ void bch2_inode_init_early(struct bch_fs *c,
 	get_random_bytes(&inode_u->bi_hash_seed, sizeof(inode_u->bi_hash_seed));
 }
 
-void bch2_inode_init_late(struct bch_inode_unpacked *inode_u, u64 now,
+void bch2_inode_init_late(struct bch_fs *c,
+			  struct bch_inode_unpacked *inode_u, u64 now,
 			  uid_t uid, gid_t gid, umode_t mode, dev_t rdev,
 			  struct bch_inode_unpacked *parent)
 {
@@ -934,6 +935,9 @@ void bch2_inode_init_late(struct bch_inode_unpacked *inode_u, u64 now,
 
 	if (!S_ISDIR(mode))
 		inode_u->bi_casefold = 0;
+
+	if (bch2_inode_casefold(c, inode_u))
+		inode_u->bi_flags |= BCH_INODE_has_case_insensitive;
 }
 
 void bch2_inode_init(struct bch_fs *c, struct bch_inode_unpacked *inode_u,
@@ -941,7 +945,7 @@ void bch2_inode_init(struct bch_fs *c, struct bch_inode_unpacked *inode_u,
 		     struct bch_inode_unpacked *parent)
 {
 	bch2_inode_init_early(c, inode_u);
-	bch2_inode_init_late(inode_u, bch2_current_time(c),
+	bch2_inode_init_late(c, inode_u, bch2_current_time(c),
 			     uid, gid, mode, rdev, parent);
 }
 
diff --git a/fs/bcachefs/inode.h b/fs/bcachefs/inode.h
index 851721d88295..f562ba516191 100644
--- a/fs/bcachefs/inode.h
+++ b/fs/bcachefs/inode.h
@@ -164,7 +164,7 @@ int bch2_fsck_write_inode(struct btree_trans *, struct bch_inode_unpacked *);
 
 void bch2_inode_init_early(struct bch_fs *,
 			   struct bch_inode_unpacked *);
-void bch2_inode_init_late(struct bch_inode_unpacked *, u64,
+void bch2_inode_init_late(struct bch_fs *, struct bch_inode_unpacked *, u64,
 			  uid_t, gid_t, umode_t, dev_t,
 			  struct bch_inode_unpacked *);
 void bch2_inode_init(struct bch_fs *, struct bch_inode_unpacked *,
diff --git a/fs/bcachefs/inode_format.h b/fs/bcachefs/inode_format.h
index 87e193e8ed25..1f00938b1bdc 100644
--- a/fs/bcachefs/inode_format.h
+++ b/fs/bcachefs/inode_format.h
@@ -129,6 +129,10 @@ enum inode_opt_id {
 	Inode_opt_nr,
 };
 
+/*
+ * BCH_INODE_has_case_insensitive is set if any descendent is case insensitive -
+ * for overlayfs
+ */
 #define BCH_INODE_FLAGS()			\
 	x(sync,				0)	\
 	x(immutable,			1)	\
@@ -139,7 +143,8 @@ enum inode_opt_id {
 	x(i_sectors_dirty,		6)	\
 	x(unlinked,			7)	\
 	x(backptr_untrusted,		8)	\
-	x(has_child_snapshot,		9)
+	x(has_child_snapshot,		9)	\
+	x(has_case_insensitive,		10)
 
 /* bits 20+ reserved for packed fields below: */
 
diff --git a/fs/bcachefs/namei.c b/fs/bcachefs/namei.c
index 561dc034df21..148615f6952a 100644
--- a/fs/bcachefs/namei.c
+++ b/fs/bcachefs/namei.c
@@ -11,6 +11,14 @@
 
 #include <linux/posix_acl.h>
 
+static inline subvol_inum parent_inum(subvol_inum inum, struct bch_inode_unpacked *inode)
+{
+	return (subvol_inum) {
+		.subvol	= inode->bi_parent_subvol ?: inum.subvol,
+		.inum	= inode->bi_dir,
+	};
+}
+
 static inline int is_subdir_for_nlink(struct bch_inode_unpacked *inode)
 {
 	return S_ISDIR(inode->bi_mode) && !inode->bi_subvol;
@@ -49,7 +57,7 @@ int bch2_create_trans(struct btree_trans *trans,
 
 	if (!(flags & BCH_CREATE_SNAPSHOT)) {
 		/* Normal create path - allocate a new inode: */
-		bch2_inode_init_late(new_inode, now, uid, gid, mode, rdev, dir_u);
+		bch2_inode_init_late(c, new_inode, now, uid, gid, mode, rdev, dir_u);
 
 		if (flags & BCH_CREATE_TMPFILE)
 			new_inode->bi_flags |= BCH_INODE_unlinked;
@@ -512,6 +520,13 @@ int bch2_rename_trans(struct btree_trans *trans,
 			goto err;
 		}
 
+		ret =   bch2_maybe_propagate_has_case_insensitive(trans, src_inum, src_inode_u) ?:
+			(mode == BCH_RENAME_EXCHANGE
+			 ? bch2_maybe_propagate_has_case_insensitive(trans, dst_inum, dst_inode_u)
+			 : 0);
+		if (ret)
+			goto err;
+
 		if (is_subdir_for_nlink(src_inode_u)) {
 			src_dir_u->bi_nlink--;
 			dst_dir_u->bi_nlink++;
@@ -613,8 +628,7 @@ int bch2_inum_to_path(struct btree_trans *trans, subvol_inum inum, struct printb
 			goto disconnected;
 		}
 
-		inum.subvol	= inode.bi_parent_subvol ?: inum.subvol;
-		inum.inum	= inode.bi_dir;
+		inum = parent_inum(inum, &inode);
 
 		u32 snapshot;
 		ret = bch2_subvolume_get_snapshot(trans, inum.subvol, &snapshot);
@@ -849,3 +863,149 @@ int __bch2_check_dirent_target(struct btree_trans *trans,
 	bch_err_fn(c, ret);
 	return ret;
 }
+
+/*
+ * BCH_INODE_has_case_insensitive:
+ * We have to track whether directories have any descendent directory that is
+ * casefolded - for overlayfs:
+ */
+
+static int bch2_propagate_has_case_insensitive(struct btree_trans *trans, subvol_inum inum)
+{
+	struct btree_iter iter = {};
+	int ret = 0;
+
+	while (true) {
+		struct bch_inode_unpacked inode;
+		ret = bch2_inode_peek(trans, &iter, &inode, inum,
+				      BTREE_ITER_intent|BTREE_ITER_with_updates);
+		if (ret)
+			break;
+
+		if (inode.bi_flags & BCH_INODE_has_case_insensitive)
+			break;
+
+		inode.bi_flags |= BCH_INODE_has_case_insensitive;
+		ret = bch2_inode_write(trans, &iter, &inode);
+		if (ret)
+			break;
+
+		bch2_trans_iter_exit(trans, &iter);
+		if (subvol_inum_eq(inum, BCACHEFS_ROOT_SUBVOL_INUM))
+			break;
+
+		inum = parent_inum(inum, &inode);
+	}
+
+	bch2_trans_iter_exit(trans, &iter);
+	return ret;
+}
+
+int bch2_maybe_propagate_has_case_insensitive(struct btree_trans *trans, subvol_inum inum,
+					      struct bch_inode_unpacked *inode)
+{
+	if (!bch2_inode_casefold(trans->c, inode))
+		return 0;
+
+	inode->bi_flags |= BCH_INODE_has_case_insensitive;
+
+	return bch2_propagate_has_case_insensitive(trans, parent_inum(inum, inode));
+}
+
+int bch2_check_inode_has_case_insensitive(struct btree_trans *trans,
+					  struct bch_inode_unpacked *inode,
+					  snapshot_id_list *snapshot_overwrites,
+					  bool *do_update)
+{
+	struct printbuf buf = PRINTBUF;
+	bool repairing_parents = false;
+	int ret = 0;
+
+	if (!S_ISDIR(inode->bi_mode)) {
+		/*
+		 * Old versions set bi_casefold for non dirs, but that's
+		 * unnecessary and wasteful
+		 */
+		if (inode->bi_casefold) {
+			inode->bi_casefold = 0;
+			*do_update = true;
+		}
+		return 0;
+	}
+
+	if (trans->c->sb.version < bcachefs_metadata_version_inode_has_case_insensitive)
+		return 0;
+
+	if (bch2_inode_casefold(trans->c, inode) &&
+	    !(inode->bi_flags & BCH_INODE_has_case_insensitive)) {
+		prt_printf(&buf, "casefolded dir with has_case_insensitive not set\ninum %llu:%u ",
+			   inode->bi_inum, inode->bi_snapshot);
+
+		ret = bch2_inum_snapshot_to_path(trans, inode->bi_inum, inode->bi_snapshot,
+						 snapshot_overwrites, &buf);
+		if (ret)
+			goto err;
+
+		if (fsck_err(trans, inode_has_case_insensitive_not_set, "%s", buf.buf)) {
+			inode->bi_flags |= BCH_INODE_has_case_insensitive;
+			*do_update = true;
+		}
+	}
+
+	if (!(inode->bi_flags & BCH_INODE_has_case_insensitive))
+		goto out;
+
+	struct bch_inode_unpacked dir = *inode;
+	u32 snapshot = dir.bi_snapshot;
+
+	while (!(dir.bi_inum	== BCACHEFS_ROOT_INO &&
+		 dir.bi_subvol	== BCACHEFS_ROOT_SUBVOL)) {
+		if (dir.bi_parent_subvol) {
+			ret = bch2_subvolume_get_snapshot(trans, dir.bi_parent_subvol, &snapshot);
+			if (ret)
+				goto err;
+
+			snapshot_overwrites = NULL;
+		}
+
+		ret = bch2_inode_find_by_inum_snapshot(trans, dir.bi_dir, snapshot, &dir, 0);
+		if (ret)
+			goto err;
+
+		if (!(dir.bi_flags & BCH_INODE_has_case_insensitive)) {
+			prt_printf(&buf, "parent of casefolded dir with has_case_insensitive not set\n");
+
+			ret = bch2_inum_snapshot_to_path(trans, dir.bi_inum, dir.bi_snapshot,
+							 snapshot_overwrites, &buf);
+			if (ret)
+				goto err;
+
+			if (fsck_err(trans, inode_parent_has_case_insensitive_not_set, "%s", buf.buf)) {
+				dir.bi_flags |= BCH_INODE_has_case_insensitive;
+				ret = __bch2_fsck_write_inode(trans, &dir);
+				if (ret)
+					goto err;
+			}
+		}
+
+		/*
+		 * We only need to check the first parent, unless we find an
+		 * inconsistency
+		 */
+		if (!repairing_parents)
+			break;
+	}
+out:
+err:
+fsck_err:
+	printbuf_exit(&buf);
+	if (ret)
+		return ret;
+
+	if (repairing_parents) {
+		return bch2_trans_commit(trans, NULL, NULL, BCH_TRANS_COMMIT_no_enospc) ?:
+			-BCH_ERR_transaction_restart_nested;
+	}
+
+	return 0;
+}
diff --git a/fs/bcachefs/namei.h b/fs/bcachefs/namei.h
index d4d2d2d69517..ae6ebc2d0785 100644
--- a/fs/bcachefs/namei.h
+++ b/fs/bcachefs/namei.h
@@ -71,4 +71,9 @@ static inline int bch2_check_dirent_target(struct btree_trans *trans,
 	return __bch2_check_dirent_target(trans, dirent_iter, d, target, in_fsck);
 }
 
+int bch2_maybe_propagate_has_case_insensitive(struct btree_trans *, subvol_inum,
+					      struct bch_inode_unpacked *);
+int bch2_check_inode_has_case_insensitive(struct btree_trans *, struct bch_inode_unpacked *,
+					  snapshot_id_list *, bool *);
+
 #endif /* _BCACHEFS_NAMEI_H */
diff --git a/fs/bcachefs/sb-downgrade.c b/fs/bcachefs/sb-downgrade.c
index 296c6c925386..861fce1630f0 100644
--- a/fs/bcachefs/sb-downgrade.c
+++ b/fs/bcachefs/sb-downgrade.c
@@ -100,7 +100,11 @@
 	  BCH_FSCK_ERR_ptr_to_missing_backpointer)		\
 	x(stripe_backpointers,					\
 	  BIT_ULL(BCH_RECOVERY_PASS_check_extents_to_backpointers),\
-	  BCH_FSCK_ERR_ptr_to_missing_backpointer)
+	  BCH_FSCK_ERR_ptr_to_missing_backpointer)		\
+	x(inode_has_case_insensitive,				\
+	  BIT_ULL(BCH_RECOVERY_PASS_check_inodes),		\
+	  BCH_FSCK_ERR_inode_has_case_insensitive_not_set,	\
+	  BCH_FSCK_ERR_inode_parent_has_case_insensitive_not_set)
 
 #define DOWNGRADE_TABLE()					\
 	x(bucket_stripe_sectors,				\
diff --git a/fs/bcachefs/sb-errors_format.h b/fs/bcachefs/sb-errors_format.h
index 68d99ca70634..8f6b02571df0 100644
--- a/fs/bcachefs/sb-errors_format.h
+++ b/fs/bcachefs/sb-errors_format.h
@@ -239,6 +239,8 @@ enum bch_fsck_flags {
 	x(inode_unreachable,					210,	FSCK_AUTOFIX)	\
 	x(inode_journal_seq_in_future,				299,	FSCK_AUTOFIX)	\
 	x(inode_i_sectors_underflow,				312,	FSCK_AUTOFIX)	\
+	x(inode_has_case_insensitive_not_set,			316,	FSCK_AUTOFIX)	\
+	x(inode_parent_has_case_insensitive_not_set,		317,	FSCK_AUTOFIX)	\
 	x(vfs_inode_i_blocks_underflow,				311,	FSCK_AUTOFIX)	\
 	x(vfs_inode_i_blocks_not_zero_at_truncate,		313,	FSCK_AUTOFIX)	\
 	x(deleted_inode_but_clean,				211,	FSCK_AUTOFIX)	\
@@ -325,7 +327,7 @@ enum bch_fsck_flags {
 	x(dirent_stray_data_after_cf_name,			305,	0)		\
 	x(rebalance_work_incorrectly_set,			309,	FSCK_AUTOFIX)	\
 	x(rebalance_work_incorrectly_unset,			310,	FSCK_AUTOFIX)	\
-	x(MAX,							316,	0)
+	x(MAX,							318,	0)
 
 enum bch_sb_error_id {
 #define x(t, n, ...) BCH_FSCK_ERR_##t = n,
-- 
2.49.0

[PATCH 2/6] darray: lift from bcachefs

[Kent Overstreet]

dynamic arrays - inspired from CCAN darrays, basically c++ stl vectors.

Open coding these is unwieldy. In this series they're used by the new
dcache code for exclusion between case insensitive directories and
overlayfs, and we've got a lot of open coded dynamic arrays that could
be cleaned up with this.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
---
 MAINTAINERS                             |  7 +++
 fs/bcachefs/Makefile                    |  1 -
 fs/bcachefs/btree_node_scan_types.h     |  2 +-
 fs/bcachefs/btree_types.h               |  2 +-
 fs/bcachefs/btree_update.c              |  1 +
 fs/bcachefs/btree_write_buffer_types.h  |  2 +-
 fs/bcachefs/disk_accounting_types.h     |  2 +-
 fs/bcachefs/fsck.c                      |  2 +-
 fs/bcachefs/journal_io.h                |  2 +-
 fs/bcachefs/journal_sb.c                |  2 +-
 fs/bcachefs/rcu_pending.c               |  3 +-
 fs/bcachefs/sb-downgrade.c              |  3 +-
 fs/bcachefs/sb-errors_types.h           |  2 +-
 fs/bcachefs/sb-members.h                |  3 +-
 fs/bcachefs/snapshot_types.h            |  3 +-
 fs/bcachefs/subvolume.h                 |  1 -
 fs/bcachefs/thread_with_file_types.h    |  2 +-
 fs/bcachefs/util.h                      | 28 +---------
 {fs/bcachefs => include/linux}/darray.h | 70 ++++++++++++++-----------
 include/linux/darray_types.h            | 33 ++++++++++++
 lib/Makefile                            |  2 +-
 {fs/bcachefs => lib}/darray.c           |  9 +++-
 22 files changed, 106 insertions(+), 76 deletions(-)
 rename {fs/bcachefs => include/linux}/darray.h (64%)
 create mode 100644 include/linux/darray_types.h
 rename {fs/bcachefs => lib}/darray.c (75%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 3cbf9ac0d83f..ae460234af14 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6515,6 +6515,13 @@ F:	net/ax25/ax25_out.c
 F:	net/ax25/ax25_timer.c
 F:	net/ax25/sysctl_net_ax25.c
 
+DARRAY
+M:	Kent Overstreet <kent.overstreet@linux.dev>
+L:	linux-bcachefs@vger.kernel.org
+S:	Maintained
+F:	include/linux/darray.h
+F:	include/linux/darray_types.h
+
 DATA ACCESS MONITOR
 M:	SeongJae Park <sj@kernel.org>
 L:	damon@lists.linux.dev
diff --git a/fs/bcachefs/Makefile b/fs/bcachefs/Makefile
index 93c8ee5425c8..d71621711cfa 100644
--- a/fs/bcachefs/Makefile
+++ b/fs/bcachefs/Makefile
@@ -28,7 +28,6 @@ bcachefs-y		:=	\
 	checksum.o		\
 	clock.o			\
 	compress.o		\
-	darray.o		\
 	data_update.o		\
 	debug.o			\
 	dirent.o		\
diff --git a/fs/bcachefs/btree_node_scan_types.h b/fs/bcachefs/btree_node_scan_types.h
index 2811b6857c97..422d49a5c57c 100644
--- a/fs/bcachefs/btree_node_scan_types.h
+++ b/fs/bcachefs/btree_node_scan_types.h
@@ -2,7 +2,7 @@
 #ifndef _BCACHEFS_BTREE_NODE_SCAN_TYPES_H
 #define _BCACHEFS_BTREE_NODE_SCAN_TYPES_H
 
-#include "darray.h"
+#include <linux/darray.h>
 
 struct found_btree_node {
 	bool			range_updated:1;
diff --git a/fs/bcachefs/btree_types.h b/fs/bcachefs/btree_types.h
index 6a0b82886581..7020ceef50e7 100644
--- a/fs/bcachefs/btree_types.h
+++ b/fs/bcachefs/btree_types.h
@@ -2,13 +2,13 @@
 #ifndef _BCACHEFS_BTREE_TYPES_H
 #define _BCACHEFS_BTREE_TYPES_H
 
+#include <linux/darray_types.h>
 #include <linux/list.h>
 #include <linux/rhashtable.h>
 
 #include "bbpos_types.h"
 #include "btree_key_cache_types.h"
 #include "buckets_types.h"
-#include "darray.h"
 #include "errcode.h"
 #include "journal_types.h"
 #include "replicas_types.h"
diff --git a/fs/bcachefs/btree_update.c b/fs/bcachefs/btree_update.c
index 20fba8d17431..afd05c3dfd03 100644
--- a/fs/bcachefs/btree_update.c
+++ b/fs/bcachefs/btree_update.c
@@ -14,6 +14,7 @@
 #include "snapshot.h"
 #include "trace.h"
 
+#include <linux/darray.h>
 #include <linux/string_helpers.h>
 
 static inline int btree_insert_entry_cmp(const struct btree_insert_entry *l,
diff --git a/fs/bcachefs/btree_write_buffer_types.h b/fs/bcachefs/btree_write_buffer_types.h
index e9e76e20f43b..d39d163c6ea9 100644
--- a/fs/bcachefs/btree_write_buffer_types.h
+++ b/fs/bcachefs/btree_write_buffer_types.h
@@ -2,7 +2,7 @@
 #ifndef _BCACHEFS_BTREE_WRITE_BUFFER_TYPES_H
 #define _BCACHEFS_BTREE_WRITE_BUFFER_TYPES_H
 
-#include "darray.h"
+#include <linux/darray_types.h>
 #include "journal_types.h"
 
 #define BTREE_WRITE_BUFERED_VAL_U64s_MAX	4
diff --git a/fs/bcachefs/disk_accounting_types.h b/fs/bcachefs/disk_accounting_types.h
index b1982131b206..242b3270cd5c 100644
--- a/fs/bcachefs/disk_accounting_types.h
+++ b/fs/bcachefs/disk_accounting_types.h
@@ -2,7 +2,7 @@
 #ifndef _BCACHEFS_DISK_ACCOUNTING_TYPES_H
 #define _BCACHEFS_DISK_ACCOUNTING_TYPES_H
 
-#include "darray.h"
+#include <linux/darray.h>
 
 struct accounting_mem_entry {
 	struct bpos				pos;
diff --git a/fs/bcachefs/fsck.c b/fs/bcachefs/fsck.c
index 338309d0a570..cc56c706d2b0 100644
--- a/fs/bcachefs/fsck.c
+++ b/fs/bcachefs/fsck.c
@@ -6,7 +6,6 @@
 #include "btree_cache.h"
 #include "btree_update.h"
 #include "buckets.h"
-#include "darray.h"
 #include "dirent.h"
 #include "error.h"
 #include "fs.h"
@@ -21,6 +20,7 @@
 #include "xattr.h"
 
 #include <linux/bsearch.h>
+#include <linux/darray.h>
 #include <linux/dcache.h> /* struct qstr */
 
 static int dirent_points_to_inode_nowarn(struct bkey_s_c_dirent d,
diff --git a/fs/bcachefs/journal_io.h b/fs/bcachefs/journal_io.h
index 12b39fcb4424..ffa543424e9e 100644
--- a/fs/bcachefs/journal_io.h
+++ b/fs/bcachefs/journal_io.h
@@ -2,7 +2,7 @@
 #ifndef _BCACHEFS_JOURNAL_IO_H
 #define _BCACHEFS_JOURNAL_IO_H
 
-#include "darray.h"
+#include <linux/darray_types.h>
 
 void bch2_journal_pos_from_member_info_set(struct bch_fs *);
 void bch2_journal_pos_from_member_info_resume(struct bch_fs *);
diff --git a/fs/bcachefs/journal_sb.c b/fs/bcachefs/journal_sb.c
index 62b910f2fb27..68b960e08f12 100644
--- a/fs/bcachefs/journal_sb.c
+++ b/fs/bcachefs/journal_sb.c
@@ -2,8 +2,8 @@
 
 #include "bcachefs.h"
 #include "journal_sb.h"
-#include "darray.h"
 
+#include <linux/darray.h>
 #include <linux/sort.h>
 
 /* BCH_SB_FIELD_journal: */
diff --git a/fs/bcachefs/rcu_pending.c b/fs/bcachefs/rcu_pending.c
index bef2aa1b8bcd..2cf3d55d0bbc 100644
--- a/fs/bcachefs/rcu_pending.c
+++ b/fs/bcachefs/rcu_pending.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 #define pr_fmt(fmt) "%s() " fmt "\n", __func__
 
+#include <linux/darray.h>
 #include <linux/generic-radix-tree.h>
 #include <linux/mm.h>
 #include <linux/percpu.h>
@@ -9,8 +10,6 @@
 #include <linux/vmalloc.h>
 
 #include "rcu_pending.h"
-#include "darray.h"
-#include "util.h"
 
 #define static_array_for_each(_a, _i)			\
 	for (typeof(&(_a)[0]) _i = _a;			\
diff --git a/fs/bcachefs/sb-downgrade.c b/fs/bcachefs/sb-downgrade.c
index 861fce1630f0..d4e22c43eeb2 100644
--- a/fs/bcachefs/sb-downgrade.c
+++ b/fs/bcachefs/sb-downgrade.c
@@ -6,12 +6,13 @@
  */
 
 #include "bcachefs.h"
-#include "darray.h"
 #include "recovery_passes.h"
 #include "sb-downgrade.h"
 #include "sb-errors.h"
 #include "super-io.h"
 
+#include <linux/darray.h>
+
 #define RECOVERY_PASS_ALL_FSCK		BIT_ULL(63)
 
 /*
diff --git a/fs/bcachefs/sb-errors_types.h b/fs/bcachefs/sb-errors_types.h
index 40325239c3b0..3b28871d23ed 100644
--- a/fs/bcachefs/sb-errors_types.h
+++ b/fs/bcachefs/sb-errors_types.h
@@ -2,7 +2,7 @@
 #ifndef _BCACHEFS_SB_ERRORS_TYPES_H
 #define _BCACHEFS_SB_ERRORS_TYPES_H
 
-#include "darray.h"
+#include <linux/darray_types.h>
 
 struct bch_sb_error_entry_cpu {
 	u64			id:16,
diff --git a/fs/bcachefs/sb-members.h b/fs/bcachefs/sb-members.h
index 6bd9b86aee5b..09b751a75020 100644
--- a/fs/bcachefs/sb-members.h
+++ b/fs/bcachefs/sb-members.h
@@ -2,10 +2,11 @@
 #ifndef _BCACHEFS_SB_MEMBERS_H
 #define _BCACHEFS_SB_MEMBERS_H
 
-#include "darray.h"
 #include "bkey_types.h"
 #include "enumerated_ref.h"
 
+#include <linux/darray.h>
+
 extern char * const bch2_member_error_strs[];
 
 static inline struct bch_member *
diff --git a/fs/bcachefs/snapshot_types.h b/fs/bcachefs/snapshot_types.h
index 0ab698f13e5c..31f96d1cf5f4 100644
--- a/fs/bcachefs/snapshot_types.h
+++ b/fs/bcachefs/snapshot_types.h
@@ -3,9 +3,10 @@
 #define _BCACHEFS_SNAPSHOT_TYPES_H
 
 #include "bbpos_types.h"
-#include "darray.h"
 #include "subvolume_types.h"
 
+#include <linux/darray_types.h>
+
 typedef DARRAY(u32) snapshot_id_list;
 
 #define IS_ANCESTOR_BITMAP	128
diff --git a/fs/bcachefs/subvolume.h b/fs/bcachefs/subvolume.h
index 075f55e25c70..771ade03a348 100644
--- a/fs/bcachefs/subvolume.h
+++ b/fs/bcachefs/subvolume.h
@@ -2,7 +2,6 @@
 #ifndef _BCACHEFS_SUBVOLUME_H
 #define _BCACHEFS_SUBVOLUME_H
 
-#include "darray.h"
 #include "subvolume_types.h"
 
 int bch2_check_subvols(struct bch_fs *);
diff --git a/fs/bcachefs/thread_with_file_types.h b/fs/bcachefs/thread_with_file_types.h
index f4d484d44f63..254b8493ec4b 100644
--- a/fs/bcachefs/thread_with_file_types.h
+++ b/fs/bcachefs/thread_with_file_types.h
@@ -2,7 +2,7 @@
 #ifndef _BCACHEFS_THREAD_WITH_FILE_TYPES_H
 #define _BCACHEFS_THREAD_WITH_FILE_TYPES_H
 
-#include "darray.h"
+#include <linux/darray_types.h>
 
 struct stdio_buf {
 	spinlock_t		lock;
diff --git a/fs/bcachefs/util.h b/fs/bcachefs/util.h
index 25cf61ebd40c..3985636cbc24 100644
--- a/fs/bcachefs/util.h
+++ b/fs/bcachefs/util.h
@@ -5,24 +5,24 @@
 #include <linux/bio.h>
 #include <linux/blkdev.h>
 #include <linux/closure.h>
+#include <linux/darray.h>
 #include <linux/errno.h>
 #include <linux/freezer.h>
 #include <linux/kernel.h>
 #include <linux/min_heap.h>
-#include <linux/sched/clock.h>
 #include <linux/llist.h>
 #include <linux/log2.h>
 #include <linux/percpu.h>
 #include <linux/preempt.h>
 #include <linux/random.h>
 #include <linux/ratelimit.h>
+#include <linux/sched/clock.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/workqueue.h>
 
 #include "mean_and_variance.h"
 
-#include "darray.h"
 #include "time_stats.h"
 
 struct closure;
@@ -552,30 +552,6 @@ static inline void memset_u64s_tail(void *s, int c, unsigned bytes)
 	memset(s + bytes, c, rem);
 }
 
-/* just the memmove, doesn't update @_nr */
-#define __array_insert_item(_array, _nr, _pos)				\
-	memmove(&(_array)[(_pos) + 1],					\
-		&(_array)[(_pos)],					\
-		sizeof((_array)[0]) * ((_nr) - (_pos)))
-
-#define array_insert_item(_array, _nr, _pos, _new_item)			\
-do {									\
-	__array_insert_item(_array, _nr, _pos);				\
-	(_nr)++;							\
-	(_array)[(_pos)] = (_new_item);					\
-} while (0)
-
-#define array_remove_items(_array, _nr, _pos, _nr_to_remove)		\
-do {									\
-	(_nr) -= (_nr_to_remove);					\
-	memmove(&(_array)[(_pos)],					\
-		&(_array)[(_pos) + (_nr_to_remove)],			\
-		sizeof((_array)[0]) * ((_nr) - (_pos)));		\
-} while (0)
-
-#define array_remove_item(_array, _nr, _pos)				\
-	array_remove_items(_array, _nr, _pos, 1)
-
 static inline void __move_gap(void *array, size_t element_size,
 			      size_t nr, size_t size,
 			      size_t old_gap, size_t new_gap)
diff --git a/fs/bcachefs/darray.h b/include/linux/darray.h
similarity index 64%
rename from fs/bcachefs/darray.h
rename to include/linux/darray.h
index 50ec3decfe8c..7a0c0159b319 100644
--- a/fs/bcachefs/darray.h
+++ b/include/linux/darray.h
@@ -1,45 +1,26 @@
 /* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _BCACHEFS_DARRAY_H
-#define _BCACHEFS_DARRAY_H
+/*
+ * (C) 2022-2024 Kent Overstreet <kent.overstreet@linux.dev>
+ */
+#ifndef _LINUX_DARRAY_H
+#define _LINUX_DARRAY_H
 
 /*
- * Dynamic arrays:
+ * Dynamic arrays
  *
  * Inspired by CCAN's darray
  */
 
+#include <linux/darray_types.h>
 #include <linux/slab.h>
 
-#define DARRAY_PREALLOCATED(_type, _nr)					\
-struct {								\
-	size_t nr, size;						\
-	_type *data;							\
-	_type preallocated[_nr];					\
-}
-
-#define DARRAY(_type) DARRAY_PREALLOCATED(_type, 0)
-
-typedef DARRAY(char)	darray_char;
-typedef DARRAY(char *)	darray_str;
-typedef DARRAY(const char *) darray_const_str;
-
-typedef DARRAY(u8)	darray_u8;
-typedef DARRAY(u16)	darray_u16;
-typedef DARRAY(u32)	darray_u32;
-typedef DARRAY(u64)	darray_u64;
-
-typedef DARRAY(s8)	darray_s8;
-typedef DARRAY(s16)	darray_s16;
-typedef DARRAY(s32)	darray_s32;
-typedef DARRAY(s64)	darray_s64;
-
-int __bch2_darray_resize_noprof(darray_char *, size_t, size_t, gfp_t);
+int __darray_resize_slowpath(darray_char *, size_t, size_t, gfp_t);
 
 #define __bch2_darray_resize(...)	alloc_hooks(__bch2_darray_resize_noprof(__VA_ARGS__))
 
 #define __darray_resize(_d, _element_size, _new_size, _gfp)		\
 	(unlikely((_new_size) > (_d)->size)				\
-	 ? __bch2_darray_resize((_d), (_element_size), (_new_size), (_gfp))\
+	 ? __darray_resize_slowpath((_d), (_element_size), (_new_size), (_gfp))\
 	 : 0)
 
 #define darray_resize_gfp(_d, _new_size, _gfp)				\
@@ -74,6 +55,28 @@ int __bch2_darray_resize_noprof(darray_char *, size_t, size_t, gfp_t);
 #define darray_first(_d)	((_d).data[0])
 #define darray_last(_d)		((_d).data[(_d).nr - 1])
 
+/* Insert/remove items into the middle of a darray: */
+
+#define array_insert_item(_array, _nr, _pos, _new_item)			\
+do {									\
+	memmove(&(_array)[(_pos) + 1],					\
+		&(_array)[(_pos)],					\
+		sizeof((_array)[0]) * ((_nr) - (_pos)));		\
+	(_nr)++;							\
+	(_array)[(_pos)] = (_new_item);					\
+} while (0)
+
+#define array_remove_items(_array, _nr, _pos, _nr_to_remove)		\
+do {									\
+	(_nr) -= (_nr_to_remove);					\
+	memmove(&(_array)[(_pos)],					\
+		&(_array)[(_pos) + (_nr_to_remove)],			\
+		sizeof((_array)[0]) * ((_nr) - (_pos)));		\
+} while (0)
+
+#define array_remove_item(_array, _nr, _pos)				\
+	array_remove_items(_array, _nr, _pos, 1)
+
 #define darray_insert_item(_d, pos, _item)				\
 ({									\
 	size_t _pos = (pos);						\
@@ -84,10 +87,15 @@ int __bch2_darray_resize_noprof(darray_char *, size_t, size_t, gfp_t);
 	_ret;								\
 })
 
+#define darray_remove_items(_d, _pos, _nr_to_remove)			\
+	array_remove_items((_d)->data, (_d)->nr, (_pos) - (_d)->data, _nr_to_remove)
+
 #define darray_remove_item(_d, _pos)					\
-	array_remove_item((_d)->data, (_d)->nr, (_pos) - (_d)->data)
+	darray_remove_items(_d, _pos, 1)
+
+/* Iteration: */
 
-#define __darray_for_each(_d, _i)						\
+#define __darray_for_each(_d, _i)					\
 	for ((_i) = (_d).data; _i < (_d).data + (_d).nr; _i++)
 
 #define darray_for_each(_d, _i)						\
@@ -111,4 +119,4 @@ do {									\
 	darray_init(_d);						\
 } while (0)
 
-#endif /* _BCACHEFS_DARRAY_H */
+#endif /* _LINUX_DARRAY_H */
diff --git a/include/linux/darray_types.h b/include/linux/darray_types.h
new file mode 100644
index 000000000000..c55484487905
--- /dev/null
+++ b/include/linux/darray_types.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * (C) 2022-2024 Kent Overstreet <kent.overstreet@linux.dev>
+ */
+#ifndef _LINUX_DARRAY_TYPES_H
+#define _LINUX_DARRAY_TYPES_H
+
+#include <linux/types.h>
+
+#define DARRAY_PREALLOCATED(_type, _nr)					\
+struct {								\
+	size_t nr, size;						\
+	_type *data;							\
+	_type preallocated[_nr];					\
+}
+
+#define DARRAY(_type) DARRAY_PREALLOCATED(_type, 0)
+
+typedef DARRAY(char)	darray_char;
+typedef DARRAY(char *)	darray_str;
+typedef DARRAY(const char *) darray_const_str;
+
+typedef DARRAY(u8)	darray_u8;
+typedef DARRAY(u16)	darray_u16;
+typedef DARRAY(u32)	darray_u32;
+typedef DARRAY(u64)	darray_u64;
+
+typedef DARRAY(s8)	darray_s8;
+typedef DARRAY(s16)	darray_s16;
+typedef DARRAY(s32)	darray_s32;
+typedef DARRAY(s64)	darray_s64;
+
+#endif /* _LINUX_DARRAY_TYPES_H */
diff --git a/lib/Makefile b/lib/Makefile
index f07b24ce1b3f..4f053d577bf3 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -56,7 +56,7 @@ obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \
 	 bsearch.o find_bit.o llist.o lwq.o memweight.o kfifo.o \
 	 percpu-refcount.o rhashtable.o base64.o \
 	 once.o refcount.o rcuref.o usercopy.o errseq.o bucket_locks.o \
-	 generic-radix-tree.o bitmap-str.o
+	 generic-radix-tree.o bitmap-str.o darray.o
 obj-y += string_helpers.o
 obj-y += hexdump.o
 obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
diff --git a/fs/bcachefs/darray.c b/lib/darray.c
similarity index 75%
rename from fs/bcachefs/darray.c
rename to lib/darray.c
index e86d36d23e9e..1d3820a43e14 100644
--- a/fs/bcachefs/darray.c
+++ b/lib/darray.c
@@ -1,11 +1,15 @@
 // SPDX-License-Identifier: GPL-2.0
+/*
+ * (C) 2022-2024 Kent Overstreet <kent.overstreet@linux.dev>
+ */
 
+#include <linux/darray.h>
+#include <linux/export.h>
 #include <linux/log2.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
-#include "darray.h"
 
-int __bch2_darray_resize_noprof(darray_char *d, size_t element_size, size_t new_size, gfp_t gfp)
+int __darray_resize_slowpath(darray_char *d, size_t element_size, size_t new_size, gfp_t gfp)
 {
 	if (new_size > d->size) {
 		new_size = roundup_pow_of_two(new_size);
@@ -36,3 +40,4 @@ int __bch2_darray_resize_noprof(darray_char *d, size_t element_size, size_t new_
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(__darray_resize_slowpath);
-- 
2.49.0

[PATCH 3/6] fs: SB_CASEFOLD

[Kent Overstreet]

Add a new flag indicating that a filesystem supports casefolding.

This is better than overlayfs's current method of checking for
sb->s_encoding, which isn't reliable - XFS implements ASCII casefolding,
so it won't be set there.

It's needed for overlayfs and the new dcache exclusion code to check
"should we allow union mounts on this filesystem even though the dcache
hash/compare ops are set? and do we need the new exclusion?".

Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
---
 fs/libfs.c         | 1 +
 include/linux/fs.h | 1 +
 2 files changed, 2 insertions(+)

diff --git a/fs/libfs.c b/fs/libfs.c
index 6393d7c49ee6..d9f6ed6ec4ea 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -1952,6 +1952,7 @@ void generic_set_sb_d_ops(struct super_block *sb)
 {
 #if IS_ENABLED(CONFIG_UNICODE)
 	if (sb->s_encoding) {
+		sb->s_flags |= SB_CASEFOLD;
 		sb->s_d_op = &generic_ci_dentry_ops;
 		return;
 	}
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 016b0fe1536e..ba942cd2fea1 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1228,6 +1228,7 @@ extern int send_sigurg(struct file *file);
 #define SB_SYNCHRONOUS  BIT(4)	/* Writes are synced at once */
 #define SB_MANDLOCK     BIT(6)	/* Allow mandatory locks on an FS */
 #define SB_DIRSYNC      BIT(7)	/* Directory modifications are synchronous */
+#define SB_CASEFOLD	BIT(8)	/* Superblock supports casefolding */
 #define SB_NOATIME      BIT(10)	/* Do not update access times. */
 #define SB_NODIRATIME   BIT(11)	/* Do not update directory access times */
 #define SB_SILENT       BIT(15)
-- 
2.49.0

[PATCH 4/6] fs: dcache locking for exlusion between overlayfs, casefolding

[Kent Overstreet]

Allow casefolding to be used on the same filesystem as overlayfs.

Overlayfs cannot handle a directory that has been casefolding, so this
implements locking between enabling casefolding and overlayfs.

The underlying filesystem mus also track, for each directory, whether
casefolding has been enabled on any subdirectory, and provide this
information to the VFS with the S_NO_CASEFOLD flag.

Since we can't inflate struct dentry for this, add a separate rhashtable
that functions as a per-dentry "casefolding disabled" ref.

New helpers, for overlayfs:
- d_casefold_disabled_put()
- d_casefold_disabled_get()

These acquire and release refs that check that S_NO_CASEFOLD is set on a
directory, and prevent it from being cleared while the ref exists.

For the underlying filesystem:

- d_casefolding_enable()
- d_casefolding_enable_commit()

This is for the underlying filesystem that implements casefolding, in
settar and rename. d_casefolding_enable() checks for conflicting refs
held by overlayfs and acquires refs while the rename or setatr is being
done, d_casefolding_enable_commit() releases those refs and clears the
S_NO_CASEFOLD flag.

Implementation -

We can't inflate struct dentry for this, so references are tracked in a
separate rhashtable.

To guard against races with rename(), d_casefolding_enable() must take
refs (which in turn call dget()) on all dentries with inodes that need
S_NO_CASEFOLD flipped; these dget() refs are released by commit().

d_casefold_disabled_get/put take dget() refs for the duration of the
union mount, but this is unchanged because unionfs already holds a path
ref for the duration of the mount.

Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
---
 fs/dcache.c            | 177 +++++++++++++++++++++++++++++++++++++++++
 include/linux/dcache.h |  10 +++
 include/linux/fs.h     |   3 +
 3 files changed, 190 insertions(+)

diff --git a/fs/dcache.c b/fs/dcache.c
index bd5aa136153a..971dd90bce26 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -32,6 +32,9 @@
 #include <linux/bit_spinlock.h>
 #include <linux/rculist_bl.h>
 #include <linux/list_lru.h>
+#include <linux/rhashtable.h>
+#include <linux/darray.h>
+#include <linux/errname.h>
 #include "internal.h"
 #include "mount.h"
 
@@ -3141,6 +3144,176 @@ ino_t d_parent_ino(struct dentry *dentry)
 }
 EXPORT_SYMBOL(d_parent_ino);
 
+static DEFINE_MUTEX(no_casefold_dentries_lock);
+struct rhashtable no_casefold_dentries;
+
+enum no_casefold_dentry_ref {
+	ref_casefold_disable,
+	ref_casefold_enable,
+};
+
+struct no_casefold_dentry {
+	struct rhash_head	hash;
+	struct dentry		*dentry;
+	unsigned long		ref[2];
+};
+
+static const struct rhashtable_params no_casefold_dentries_params = {
+	.head_offset		= offsetof(struct no_casefold_dentry, hash),
+	.key_offset		= offsetof(struct no_casefold_dentry, dentry),
+	.key_len		= sizeof(struct dentry *),
+	.automatic_shrinking	= true,
+};
+
+static int no_casefold_dentry_get(struct dentry *dentry,
+				  enum no_casefold_dentry_ref ref)
+{
+	struct no_casefold_dentry *n =
+		rhashtable_lookup_fast(&no_casefold_dentries,
+				       &dentry,
+				       no_casefold_dentries_params);
+	if (n) {
+		if (n->ref[!ref])
+			return -EINVAL;
+
+		n->ref[ref]++;
+		return 0;
+	}
+
+	n = kzalloc(sizeof(*n), GFP_KERNEL);
+	if (!n)
+		return -ENOMEM;
+
+	n->dentry = dget(dentry);
+	n->ref[ref]++;
+
+	int ret = rhashtable_lookup_insert_fast(&no_casefold_dentries,
+				&n->hash, no_casefold_dentries_params);
+	if (WARN_ON(ret)) {
+		kfree(n);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void no_casefold_dentry_put(struct dentry *dentry,
+				   enum no_casefold_dentry_ref ref)
+{
+	struct no_casefold_dentry *n =
+		rhashtable_lookup_fast(&no_casefold_dentries,
+				       &dentry,
+				       no_casefold_dentries_params);
+	if (WARN_ON(!n))
+		return;
+
+	if (--n->ref[ref])
+		return;
+
+	dput(n->dentry);
+	int ret = rhashtable_remove_fast(&no_casefold_dentries,
+					 &n->hash, no_casefold_dentries_params);
+	WARN_ON(ret);
+}
+
+/**
+ * d_casefold_disabled_put - drop a "casefold disabled" ref
+ *
+ * Only for overlayfs.
+ */
+void d_casefold_disabled_put(struct dentry *dentry)
+{
+	if (!(dentry->d_sb->s_flags & SB_CASEFOLD))
+		return;
+
+	guard(mutex)(&no_casefold_dentries_lock);
+	no_casefold_dentry_put(dentry, ref_casefold_disable);
+}
+EXPORT_SYMBOL_GPL(d_casefold_disabled_put);
+
+/**
+ * d_casefold_disabled_get - attempt to disable casefold on a tree
+ *
+ * Only for overlayfs.
+ *
+ * Returns -EINVAL if casefolding is in use on any subdirectory; this must be
+ * tracked by the filesystem.
+ *
+ * On success, returns with a reference held that must be released by
+ * d_casefold_disabled_put(); this ref blocks casefold from being enabled
+ * by d_casefold_enable().
+ */
+int d_casefold_disabled_get(struct dentry *dentry)
+{
+	if (!(dentry->d_sb->s_flags & SB_CASEFOLD))
+		return 0;
+
+	guard(mutex)(&no_casefold_dentries_lock);
+
+	if (!(dentry->d_inode->i_flags & S_NO_CASEFOLD))
+		return -EINVAL;
+
+	return no_casefold_dentry_get(dentry, ref_casefold_disable);
+}
+EXPORT_SYMBOL_GPL(d_casefold_disabled_get);
+
+/**
+ * d_casefold_enable - check if casefolding may be enabled on a dentry
+ *
+ * Returns -EINVAL if casefolding has been disabled on any parent directory (by
+ * overlayfs).
+ *
+ * On success the inode will hase S_NO_CASEFOLD cleared.
+ */
+int d_casefold_enable(struct dentry *dentry, struct d_casefold_enable *e)
+{
+	struct dentry *root = dentry->d_sb->s_root;
+	int ret = 0;
+
+	guard(mutex)(&no_casefold_dentries_lock);
+
+	for (struct dentry *i = dentry;
+	     i && i->d_inode->i_flags & S_NO_CASEFOLD;
+	     i = i != root ? i->d_parent : NULL) {
+		ret = darray_push(&e->refs, i);
+		if (ret)
+			goto err;
+
+		ret = no_casefold_dentry_get(i, ref_casefold_enable);
+		if (ret) {
+			darray_pop(&e->refs);
+			goto err;
+		}
+	}
+
+	return 0;
+err:
+	darray_for_each(e->refs, i)
+		no_casefold_dentry_put(*i, ref_casefold_enable);
+	darray_exit(&e->refs);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(d_casefold_enable);
+
+void d_casefold_enable_commit(struct d_casefold_enable *e, int ret)
+{
+	guard(mutex)(&no_casefold_dentries_lock);
+
+	darray_for_each(e->refs, i) {
+		if (!ret) {
+			struct inode *inode = (*i)->d_inode;
+
+			spin_lock(&inode->i_lock);
+			inode->i_flags &= ~S_NO_CASEFOLD;
+			spin_unlock(&inode->i_lock);
+		}
+
+		no_casefold_dentry_put(*i, ref_casefold_enable);
+	}
+	darray_exit(&e->refs);
+}
+EXPORT_SYMBOL_GPL(d_casefold_enable_commit);
+
 static __initdata unsigned long dhash_entries;
 static int __init set_dhash_entries(char *str)
 {
@@ -3186,6 +3359,10 @@ static void __init dcache_init(void)
 		SLAB_RECLAIM_ACCOUNT|SLAB_PANIC|SLAB_ACCOUNT,
 		d_shortname.string);
 
+	int ret = rhashtable_init(&no_casefold_dentries, &no_casefold_dentries_params);
+	if (ret)
+		panic("error initializing no_casefold_dentries: %s\n", errname(ret));
+
 	/* Hash may have been set up in dcache_init_early */
 	if (!hashdist)
 		return;
diff --git a/include/linux/dcache.h b/include/linux/dcache.h
index e9f07e37dd6f..a386f636b15d 100644
--- a/include/linux/dcache.h
+++ b/include/linux/dcache.h
@@ -3,6 +3,7 @@
 #define __LINUX_DCACHE_H
 
 #include <linux/atomic.h>
+#include <linux/darray_types.h>
 #include <linux/list.h>
 #include <linux/math.h>
 #include <linux/rculist.h>
@@ -604,4 +605,13 @@ static inline struct dentry *d_next_sibling(const struct dentry *dentry)
 	return hlist_entry_safe(dentry->d_sib.next, struct dentry, d_sib);
 }
 
+void d_casefold_disabled_put(struct dentry *dentry);
+int d_casefold_disabled_get(struct dentry *dentry);
+
+struct d_casefold_enable {
+	DARRAY(struct dentry *)	refs;
+};
+int d_casefold_enable(struct dentry *dentry, struct d_casefold_enable *e);
+void d_casefold_enable_commit(struct d_casefold_enable *e, int ret);
+
 #endif	/* __LINUX_DCACHE_H */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index ba942cd2fea1..962a60c9b50d 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -48,6 +48,7 @@
 #include <linux/rw_hint.h>
 #include <linux/file_ref.h>
 #include <linux/unicode.h>
+#include <linux/rhashtable-types.h>
 
 #include <asm/byteorder.h>
 #include <uapi/linux/fs.h>
@@ -1440,6 +1441,7 @@ struct super_block {
 
 	struct mutex		s_sync_lock;	/* sync serialisation lock */
 
+
 	/*
 	 * Indicates how deep in a filesystem stack this SB is
 	 */
@@ -2345,6 +2347,7 @@ struct super_operations {
 #define S_CASEFOLD	(1 << 15) /* Casefolded file */
 #define S_VERITY	(1 << 16) /* Verity file (using fs/verity/) */
 #define S_KERNEL_FILE	(1 << 17) /* File is in use by the kernel (eg. fs/cachefiles) */
+#define S_NO_CASEFOLD	(1 << 18) /* Directory, and all descendents, are not casefolded */
 
 /*
  * Note that nosuid etc flags are inode-specific: setting some file-system
-- 
2.49.0

[PATCH 5/6] bcachefs: Hook up d_casefold_enable()

[Kent Overstreet]

Hook up BCH_INODE_has_case_insensitive -> S_NO_CASEFOLD, and call the
new dcache methods for exclusion with overlayfs when the flag is
changing.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
---
 fs/bcachefs/fs.c | 41 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 39 insertions(+), 2 deletions(-)

diff --git a/fs/bcachefs/fs.c b/fs/bcachefs/fs.c
index 0b5d52895e05..a02f787a6d05 100644
--- a/fs/bcachefs/fs.c
+++ b/fs/bcachefs/fs.c
@@ -68,6 +68,11 @@ static inline void bch2_inode_flags_to_vfs(struct bch_fs *c, struct bch_inode_in
 		inode->v.i_flags |= S_CASEFOLD;
 	else
 		inode->v.i_flags &= ~S_CASEFOLD;
+
+	if (inode->ei_inode.bi_flags & BCH_INODE_has_case_insensitive)
+		inode->v.i_flags &= ~S_NO_CASEFOLD;
+	else
+		inode->v.i_flags |= S_NO_CASEFOLD;
 }
 
 void bch2_inode_update_after_write(struct btree_trans *trans,
@@ -916,6 +921,8 @@ static int bch2_rename2(struct mnt_idmap *idmap,
 	struct bch_inode_info *dst_inode = to_bch_ei(dst_dentry->d_inode);
 	struct bch_inode_unpacked dst_dir_u, src_dir_u;
 	struct bch_inode_unpacked src_inode_u, dst_inode_u, *whiteout_inode_u;
+	struct d_casefold_enable casefold_enable_src = {};
+	struct d_casefold_enable casefold_enable_dst = {};
 	struct btree_trans *trans;
 	enum bch_rename_mode mode = flags & RENAME_EXCHANGE
 		? BCH_RENAME_EXCHANGE
@@ -940,6 +947,21 @@ static int bch2_rename2(struct mnt_idmap *idmap,
 			 src_inode,
 			 dst_inode);
 
+	if (src_dir != dst_dir) {
+		if (bch2_inode_casefold(c, &src_inode->ei_inode)) {
+			ret = d_casefold_enable(dst_dentry, &casefold_enable_dst);
+			if (ret)
+				goto err;
+		}
+
+		if (mode == BCH_RENAME_EXCHANGE &&
+		    bch2_inode_casefold(c, &dst_inode->ei_inode)) {
+			ret = d_casefold_enable(src_dentry, &casefold_enable_src);
+			if (ret)
+				goto err;
+		}
+	}
+
 	trans = bch2_trans_get(c);
 
 	ret   = bch2_subvol_is_ro_trans(trans, src_dir->ei_inum.subvol) ?:
@@ -1044,6 +1066,9 @@ static int bch2_rename2(struct mnt_idmap *idmap,
 			   src_inode,
 			   dst_inode);
 
+	d_casefold_enable_commit(&casefold_enable_dst, ret);
+	d_casefold_enable_commit(&casefold_enable_src, ret);
+
 	return bch2_err_class(ret);
 }
 
@@ -1710,6 +1735,7 @@ static int bch2_fileattr_set(struct mnt_idmap *idmap,
 	struct bch_inode_info *inode = to_bch_ei(d_inode(dentry));
 	struct bch_fs *c = inode->v.i_sb->s_fs_info;
 	struct flags_set s = {};
+	struct d_casefold_enable casefold_enable = {};
 	int ret;
 
 	if (fa->fsx_valid) {
@@ -1742,9 +1768,17 @@ static int bch2_fileattr_set(struct mnt_idmap *idmap,
 		s.casefold = (fa->flags & FS_CASEFOLD_FL) != 0;
 		fa->flags &= ~FS_CASEFOLD_FL;
 
+		if (s.casefold) {
+			ret = d_casefold_enable(dentry, &casefold_enable);
+			if (ret)
+				goto err;
+		}
+
 		s.flags |= map_flags_rev(bch_flags_to_uflags, fa->flags);
-		if (fa->flags)
-			return -EOPNOTSUPP;
+		if (fa->flags) {
+			ret = -EOPNOTSUPP;
+			goto err;
+		}
 	}
 
 	mutex_lock(&inode->ei_update_lock);
@@ -1755,6 +1789,9 @@ static int bch2_fileattr_set(struct mnt_idmap *idmap,
 		bch2_write_inode(c, inode, fssetxattr_inode_update_fn, &s,
 			       ATTR_CTIME);
 	mutex_unlock(&inode->ei_update_lock);
+err:
+	d_casefold_enable_commit(&casefold_enable, ret);
+
 	return ret;
 }
 
-- 
2.49.0

[PATCH 6/6] overlayfs: Support casefolded filesystems

[Kent Overstreet]

Overlayfs can now work on filesystems that support casefolding, provided
the specific subtree overlayfs is using as layers don't have casefolding
enabled.

d_casefold_disabled_get() and put() are used, which check that
casefolding is enabled nowhere on a given subtree, and get and release a
reference that prevents the filesystem from enabling casefolding on that
tree while overlayfs is in use.

We also now check the new SB_CASEFOLD superblock flag; if it's set we
allow for dcache hash and compare ops to be set, relying instead on the
new dcache methods.

Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Amir Goldstein <amir73il@gmail.com>
Cc: linux-unionfs@vger.kernel.org
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
---
 fs/overlayfs/params.c | 20 +++++++++++++++++---
 fs/overlayfs/util.c   | 19 +++++++++++++++----
 2 files changed, 32 insertions(+), 7 deletions(-)

diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
index 6759f7d040c8..ae7424e075a7 100644
--- a/fs/overlayfs/params.c
+++ b/fs/overlayfs/params.c
@@ -287,7 +287,8 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
 	 * with overlayfs.  Check explicitly to prevent post-mount
 	 * failures.
 	 */
-	if (sb_has_encoding(path->mnt->mnt_sb))
+	if ((path->mnt->mnt_sb->s_flags & SB_CASEFOLD) &&
+	    !(path->dentry->d_inode->i_flags & S_NO_CASEFOLD))
 		return invalfc(fc, "case-insensitive capable filesystem on %s not supported", name);
 
 	if (ovl_dentry_weird(path->dentry))
@@ -411,20 +412,32 @@ static int ovl_do_parse_layer(struct fs_context *fc, const char *layer_name,
 	if (!name)
 		return -ENOMEM;
 
+	if (layer != Opt_workdir &&
+	    layer != Opt_upperdir) {
+		err = d_casefold_disabled_get(layer_path->dentry);
+		if (err)
+			return err;
+	}
+
 	upper = is_upper_layer(layer);
 	err = ovl_mount_dir_check(fc, layer_path, layer, name, upper);
 	if (err)
-		return err;
+		goto err_put;
 
 	if (!upper) {
 		err = ovl_ctx_realloc_lower(fc);
 		if (err)
-			return err;
+			goto err_put;
 	}
 
 	/* Store the user provided path string in ctx to show in mountinfo */
 	ovl_add_layer(fc, layer, layer_path, &name);
 	return err;
+err_put:
+	if (layer != Opt_workdir &&
+	    layer != Opt_upperdir)
+		d_casefold_disabled_put(layer_path->dentry);
+	return err;
 }
 
 static int ovl_parse_layer(struct fs_context *fc, struct fs_parameter *param,
@@ -475,6 +488,7 @@ static void ovl_reset_lowerdirs(struct ovl_fs_context *ctx)
 	ctx->lowerdir_all = NULL;
 
 	for (size_t nr = 0; nr < ctx->nr; nr++, l++) {
+		d_casefold_disabled_put(l->path.dentry);
 		path_put(&l->path);
 		kfree(l->name);
 		l->name = NULL;
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 0819c739cc2f..c515f260032c 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -205,10 +205,21 @@ bool ovl_dentry_weird(struct dentry *dentry)
 	if (!d_can_lookup(dentry) && !d_is_file(dentry) && !d_is_symlink(dentry))
 		return true;
 
-	return dentry->d_flags & (DCACHE_NEED_AUTOMOUNT |
-				  DCACHE_MANAGE_TRANSIT |
-				  DCACHE_OP_HASH |
-				  DCACHE_OP_COMPARE);
+	if (dentry->d_flags & (DCACHE_NEED_AUTOMOUNT |
+			       DCACHE_MANAGE_TRANSIT))
+		return true;
+
+	/*
+	 * The filesystem might support casefolding, but we've already checked
+	 * that casefolding isn't present on this tree: we only need to check
+	 * for non-casefolding hash/compare ops
+	 */
+	if (!(dentry->d_sb->s_flags & SB_CASEFOLD) &&
+	    (dentry->d_flags & (DCACHE_OP_HASH |
+				DCACHE_OP_COMPARE)))
+		return true;
+
+	return false;
 }
 
 enum ovl_path_type ovl_path_type(struct dentry *dentry)
-- 
2.49.0

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> This series allows overlayfs and casefolding to safely be used on the
> same filesystem by providing exclusion to ensure that overlayfs never
> has to deal with casefolded directories.
>
> Currently, overlayfs can't be used _at all_ if a filesystem even
> supports casefolding, which is really nasty for users.
>
> Components:
>
> - filesystem has to track, for each directory, "does any _descendent_
>   have casefolding enabled"
>
> - new inode flag to pass this to VFS layer
>
> - new dcache methods for providing refs for overlayfs, and filesystem
>   methods for safely clearing this flag
>
> - new superblock flag for indicating to overlayfs & dcache "filesystem
>   supports casefolding, it's safe to use provided new dcache methods are
>   used"
>

I don't think that this is really needed.

Too bad you did not ask before going through the trouble of this implementation.

I think it is enough for overlayfs to know the THIS directory has no
casefolding.

in ovl_lookup() that returns a merged directory, ovl_dentry_weird() would
result in -EIO if any of the real directories have casefolding and we can add
another sanotify in ovl_lookup_single() that the 'base' dentry is not weird()
to cover the case of casefolder changed on an underlying reference directory.

Obviously, if any of the overlayfs layer root dirs have casefolding enabled the
mount would fail.

w.r.t enabling casefolding underneath overlayfs, overlayfs documentation says:

"Changes to underlying filesystems
---------------------------------

Changes to the underlying filesystems while part of a mounted overlay
filesystem are not allowed.  If the underlying filesystem is changed,
the behavior of the overlay is undefined, though it will not result in
a crash or deadlock."

So why is enabling casefolding on underlying layers so special that we
should have specific protection for that?

From what I remember in ext4, enabling casefolding is only allowed
on empty directories.

Is this also the case for bcachefs?

If that is the case, then the situation is even simpler -
If filesystem can singal to vfs/ovl that directory is empty (i.e. S_EMPTYDIR)
then overlayfs can ignore this dir altogether when composing
the merged directory.

But again, I don't think there is a good reason to treat this case
of changing the underlying layer specially.

Please explain if I missed anything.

Thanks,
Amir.

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
> On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > This series allows overlayfs and casefolding to safely be used on the
> > same filesystem by providing exclusion to ensure that overlayfs never
> > has to deal with casefolded directories.
> >
> > Currently, overlayfs can't be used _at all_ if a filesystem even
> > supports casefolding, which is really nasty for users.
> >
> > Components:
> >
> > - filesystem has to track, for each directory, "does any _descendent_
> >   have casefolding enabled"
> >
> > - new inode flag to pass this to VFS layer
> >
> > - new dcache methods for providing refs for overlayfs, and filesystem
> >   methods for safely clearing this flag
> >
> > - new superblock flag for indicating to overlayfs & dcache "filesystem
> >   supports casefolding, it's safe to use provided new dcache methods are
> >   used"
> >
> 
> I don't think that this is really needed.
> 
> Too bad you did not ask before going through the trouble of this implementation.
> 
> I think it is enough for overlayfs to know the THIS directory has no
> casefolding.

overlayfs works on trees, not directories...

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
> > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
> > <kent.overstreet@linux.dev> wrote:
> > >
> > > This series allows overlayfs and casefolding to safely be used on the
> > > same filesystem by providing exclusion to ensure that overlayfs never
> > > has to deal with casefolded directories.
> > >
> > > Currently, overlayfs can't be used _at all_ if a filesystem even
> > > supports casefolding, which is really nasty for users.
> > >
> > > Components:
> > >
> > > - filesystem has to track, for each directory, "does any _descendent_
> > >   have casefolding enabled"
> > >
> > > - new inode flag to pass this to VFS layer
> > >
> > > - new dcache methods for providing refs for overlayfs, and filesystem
> > >   methods for safely clearing this flag
> > >
> > > - new superblock flag for indicating to overlayfs & dcache "filesystem
> > >   supports casefolding, it's safe to use provided new dcache methods are
> > >   used"
> > >
> >
> > I don't think that this is really needed.
> >
> > Too bad you did not ask before going through the trouble of this implementation.
> >
> > I think it is enough for overlayfs to know the THIS directory has no
> > casefolding.
>
> overlayfs works on trees, not directories...

I know how overlayfs works...

I've explained why I don't think that sanitizing the entire tree is needed
for creating overlayfs over a filesystem that may enable casefolding
on some of its directories.

Thanks,
Amir.

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
> On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
> > > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
> > > <kent.overstreet@linux.dev> wrote:
> > > >
> > > > This series allows overlayfs and casefolding to safely be used on the
> > > > same filesystem by providing exclusion to ensure that overlayfs never
> > > > has to deal with casefolded directories.
> > > >
> > > > Currently, overlayfs can't be used _at all_ if a filesystem even
> > > > supports casefolding, which is really nasty for users.
> > > >
> > > > Components:
> > > >
> > > > - filesystem has to track, for each directory, "does any _descendent_
> > > >   have casefolding enabled"
> > > >
> > > > - new inode flag to pass this to VFS layer
> > > >
> > > > - new dcache methods for providing refs for overlayfs, and filesystem
> > > >   methods for safely clearing this flag
> > > >
> > > > - new superblock flag for indicating to overlayfs & dcache "filesystem
> > > >   supports casefolding, it's safe to use provided new dcache methods are
> > > >   used"
> > > >
> > >
> > > I don't think that this is really needed.
> > >
> > > Too bad you did not ask before going through the trouble of this implementation.
> > >
> > > I think it is enough for overlayfs to know the THIS directory has no
> > > casefolding.
> >
> > overlayfs works on trees, not directories...
> 
> I know how overlayfs works...
> 
> I've explained why I don't think that sanitizing the entire tree is needed
> for creating overlayfs over a filesystem that may enable casefolding
> on some of its directories.

So, you want to move error checking from mount time, where we _just_
did a massive API rework so that we can return errors in a way that
users will actually see them - to open/lookup, where all we have are a
small fixed set of error codes?

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
> > On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
> > <kent.overstreet@linux.dev> wrote:
> > >
> > > On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
> > > > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
> > > > <kent.overstreet@linux.dev> wrote:
> > > > >
> > > > > This series allows overlayfs and casefolding to safely be used on the
> > > > > same filesystem by providing exclusion to ensure that overlayfs never
> > > > > has to deal with casefolded directories.
> > > > >
> > > > > Currently, overlayfs can't be used _at all_ if a filesystem even
> > > > > supports casefolding, which is really nasty for users.
> > > > >
> > > > > Components:
> > > > >
> > > > > - filesystem has to track, for each directory, "does any _descendent_
> > > > >   have casefolding enabled"
> > > > >
> > > > > - new inode flag to pass this to VFS layer
> > > > >
> > > > > - new dcache methods for providing refs for overlayfs, and filesystem
> > > > >   methods for safely clearing this flag
> > > > >
> > > > > - new superblock flag for indicating to overlayfs & dcache "filesystem
> > > > >   supports casefolding, it's safe to use provided new dcache methods are
> > > > >   used"
> > > > >
> > > >
> > > > I don't think that this is really needed.
> > > >
> > > > Too bad you did not ask before going through the trouble of this implementation.
> > > >
> > > > I think it is enough for overlayfs to know the THIS directory has no
> > > > casefolding.
> > >
> > > overlayfs works on trees, not directories...
> >
> > I know how overlayfs works...
> >
> > I've explained why I don't think that sanitizing the entire tree is needed
> > for creating overlayfs over a filesystem that may enable casefolding
> > on some of its directories.
>
> So, you want to move error checking from mount time, where we _just_
> did a massive API rework so that we can return errors in a way that
> users will actually see them - to open/lookup, where all we have are a
> small fixed set of error codes?

That's one way of putting it.

Please explain the use case.

When is overlayfs created over a subtree that is only partially case folded?
Is that really so common that a mount time error justifies all the vfs
infrastructure involved?

Thanks,
Amir.

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 04:03:27PM +0200, Amir Goldstein wrote:
> On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
> > > On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
> > > <kent.overstreet@linux.dev> wrote:
> > > >
> > > > On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
> > > > > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
> > > > > <kent.overstreet@linux.dev> wrote:
> > > > > >
> > > > > > This series allows overlayfs and casefolding to safely be used on the
> > > > > > same filesystem by providing exclusion to ensure that overlayfs never
> > > > > > has to deal with casefolded directories.
> > > > > >
> > > > > > Currently, overlayfs can't be used _at all_ if a filesystem even
> > > > > > supports casefolding, which is really nasty for users.
> > > > > >
> > > > > > Components:
> > > > > >
> > > > > > - filesystem has to track, for each directory, "does any _descendent_
> > > > > >   have casefolding enabled"
> > > > > >
> > > > > > - new inode flag to pass this to VFS layer
> > > > > >
> > > > > > - new dcache methods for providing refs for overlayfs, and filesystem
> > > > > >   methods for safely clearing this flag
> > > > > >
> > > > > > - new superblock flag for indicating to overlayfs & dcache "filesystem
> > > > > >   supports casefolding, it's safe to use provided new dcache methods are
> > > > > >   used"
> > > > > >
> > > > >
> > > > > I don't think that this is really needed.
> > > > >
> > > > > Too bad you did not ask before going through the trouble of this implementation.
> > > > >
> > > > > I think it is enough for overlayfs to know the THIS directory has no
> > > > > casefolding.
> > > >
> > > > overlayfs works on trees, not directories...
> > >
> > > I know how overlayfs works...
> > >
> > > I've explained why I don't think that sanitizing the entire tree is needed
> > > for creating overlayfs over a filesystem that may enable casefolding
> > > on some of its directories.
> >
> > So, you want to move error checking from mount time, where we _just_
> > did a massive API rework so that we can return errors in a way that
> > users will actually see them - to open/lookup, where all we have are a
> > small fixed set of error codes?
> 
> That's one way of putting it.
> 
> Please explain the use case.
> 
> When is overlayfs created over a subtree that is only partially case folded?
> Is that really so common that a mount time error justifies all the vfs
> infrastructure involved?

Amir, you've got two widely used filesystem features that conflict and
can't be used on the same filesystem.

That's _broken_.

Users hate partitioning just for separate /boot and /home, having to
partition for different applications is horrible. And since overlay fs
is used under the hood by docker, and casefolding is used under the hood
for running Windows applications, this isn't something people can
predict in advance.

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

On Tue, May 20, 2025 at 4:12 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Tue, May 20, 2025 at 04:03:27PM +0200, Amir Goldstein wrote:
> > On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
> > <kent.overstreet@linux.dev> wrote:
> > >
> > > On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
> > > > On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
> > > > <kent.overstreet@linux.dev> wrote:
> > > > >
> > > > > On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
> > > > > > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
> > > > > > <kent.overstreet@linux.dev> wrote:
> > > > > > >
> > > > > > > This series allows overlayfs and casefolding to safely be used on the
> > > > > > > same filesystem by providing exclusion to ensure that overlayfs never
> > > > > > > has to deal with casefolded directories.
> > > > > > >
> > > > > > > Currently, overlayfs can't be used _at all_ if a filesystem even
> > > > > > > supports casefolding, which is really nasty for users.
> > > > > > >
> > > > > > > Components:
> > > > > > >
> > > > > > > - filesystem has to track, for each directory, "does any _descendent_
> > > > > > >   have casefolding enabled"
> > > > > > >
> > > > > > > - new inode flag to pass this to VFS layer
> > > > > > >
> > > > > > > - new dcache methods for providing refs for overlayfs, and filesystem
> > > > > > >   methods for safely clearing this flag
> > > > > > >
> > > > > > > - new superblock flag for indicating to overlayfs & dcache "filesystem
> > > > > > >   supports casefolding, it's safe to use provided new dcache methods are
> > > > > > >   used"
> > > > > > >
> > > > > >
> > > > > > I don't think that this is really needed.
> > > > > >
> > > > > > Too bad you did not ask before going through the trouble of this implementation.
> > > > > >
> > > > > > I think it is enough for overlayfs to know the THIS directory has no
> > > > > > casefolding.
> > > > >
> > > > > overlayfs works on trees, not directories...
> > > >
> > > > I know how overlayfs works...
> > > >
> > > > I've explained why I don't think that sanitizing the entire tree is needed
> > > > for creating overlayfs over a filesystem that may enable casefolding
> > > > on some of its directories.
> > >
> > > So, you want to move error checking from mount time, where we _just_
> > > did a massive API rework so that we can return errors in a way that
> > > users will actually see them - to open/lookup, where all we have are a
> > > small fixed set of error codes?
> >
> > That's one way of putting it.
> >
> > Please explain the use case.
> >
> > When is overlayfs created over a subtree that is only partially case folded?
> > Is that really so common that a mount time error justifies all the vfs
> > infrastructure involved?
>
> Amir, you've got two widely used filesystem features that conflict and
> can't be used on the same filesystem.
>
> That's _broken_.

Correct.

I am saying that IMO a smaller impact (and less user friendly) fix is more
appropriate way to deal with this problem.

>
> Users hate partitioning just for separate /boot and /home, having to
> partition for different applications is horrible. And since overlay fs
> is used under the hood by docker, and casefolding is used under the hood
> for running Windows applications, this isn't something people can
> predict in advance.

Right, I am not expecting users to partition by application,
but my question was this:

When is overlayfs created over a subtree that is only partially case-folded?

Obviously, docker would create overlayfs on parts of the fs
and smbd/cygwin could create a case folder subtree on another
part of the fs.
I just don't see a common use case when these sections overlap.

Perhaps I am wrong (please present real world use cases),
but my claim is that this case is not common enough and therefore,
a suboptimal EIO error from lookup is good enough to prevert crossing
over into the case folded zone by mistake, just as EIO on lookup is
enough to deal with the unsupported use case of modifying
overlayfs underlying layers with overlay is mounted.

BTW, it is not enough to claim that there is no case folding for the
entire subtree to allow the mount.
For overlayfs to allow d_hash()/d_compare() fs must claim that
these implementations are the default implementation in all subtree
or at least that all layers share the same implementation.

Thanks,
Amir.

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 04:33:14PM +0200, Amir Goldstein wrote:
> On Tue, May 20, 2025 at 4:12 PM Kent Overstreet
> > Amir, you've got two widely used filesystem features that conflict and
> > can't be used on the same filesystem.
> >
> > That's _broken_.
> 
> Correct.
> 
> I am saying that IMO a smaller impact (and less user friendly) fix is more
> appropriate way to deal with this problem.

Less user friendly is an understatement.

Obscure errors that only get reported via overloaded standard error
codes is a massive problem today, for _developers_ - have you never had
a day of swearing over trying to track down where in a massive subsystem
an -EINVAL is coming from?

It's even worse for end users that don't know to check the dmesg log.

And I support my code, so these would turn into bug reports coming
across my desk - no thanks; I already get enough weird shit from other
subsystems that I have to look at and at least triage.

> > Users hate partitioning just for separate /boot and /home, having to
> > partition for different applications is horrible. And since overlay fs
> > is used under the hood by docker, and casefolding is used under the hood
> > for running Windows applications, this isn't something people can
> > predict in advance.
> 
> Right, I am not expecting users to partition by application,
> but my question was this:
> 
> When is overlayfs created over a subtree that is only partially case-folded?
> 
> Obviously, docker would create overlayfs on parts of the fs
> and smbd/cygwin could create a case folder subtree on another
> part of the fs.
> I just don't see a common use case when these sections overlap.

Today, you cannot user docker and casefolding on _different parts of_
the same filesystem.

So yees, today users do have to partition by application, or only use
one feature or the other.

This isn't about allowing casefolding and overlayfs to fix on the same
subtree, that would be a bigger project.

> Perhaps I am wrong (please present real world use cases),
> but my claim is that this case is not common enough and therefore,
> a suboptimal EIO error from lookup is good enough to prevert crossing
> over into the case folded zone by mistake, just as EIO on lookup is
> enough to deal with the unsupported use case of modifying
> overlayfs underlying layers with overlay is mounted.
> 
> BTW, it is not enough to claim that there is no case folding for the
> entire subtree to allow the mount.
> For overlayfs to allow d_hash()/d_compare() fs must claim that
> these implementations are the default implementation in all subtree
> or at least that all layers share the same implementation.

I honestly don't know what you're claiming here.

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

On Tue, May 20, 2025 at 4:44 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Tue, May 20, 2025 at 04:33:14PM +0200, Amir Goldstein wrote:
> > On Tue, May 20, 2025 at 4:12 PM Kent Overstreet
> > > Amir, you've got two widely used filesystem features that conflict and
> > > can't be used on the same filesystem.
> > >
> > > That's _broken_.
> >
> > Correct.
> >
> > I am saying that IMO a smaller impact (and less user friendly) fix is more
> > appropriate way to deal with this problem.
>
> Less user friendly is an understatement.
>
> Obscure errors that only get reported via overloaded standard error
> codes is a massive problem today, for _developers_ - have you never had
> a day of swearing over trying to track down where in a massive subsystem
> an -EINVAL is coming from?
>
> It's even worse for end users that don't know to check the dmesg log.
>
> And I support my code, so these would turn into bug reports coming
> across my desk - no thanks; I already get enough weird shit from other
> subsystems that I have to look at and at least triage.
>
> > > Users hate partitioning just for separate /boot and /home, having to
> > > partition for different applications is horrible. And since overlay fs
> > > is used under the hood by docker, and casefolding is used under the hood
> > > for running Windows applications, this isn't something people can
> > > predict in advance.
> >
> > Right, I am not expecting users to partition by application,
> > but my question was this:
> >
> > When is overlayfs created over a subtree that is only partially case-folded?
> >
> > Obviously, docker would create overlayfs on parts of the fs
> > and smbd/cygwin could create a case folder subtree on another
> > part of the fs.
> > I just don't see a common use case when these sections overlap.
>
> Today, you cannot user docker and casefolding on _different parts of_
> the same filesystem.
>
> So yees, today users do have to partition by application, or only use
> one feature or the other.
>

Didn't say there was no problem.

Argued that your fix is a big gun and not worth the added complexity.

Let's see what Miklos thinks.

> This isn't about allowing casefolding and overlayfs to fix on the same
> subtree, that would be a bigger project.
>
> > Perhaps I am wrong (please present real world use cases),
> > but my claim is that this case is not common enough and therefore,
> > a suboptimal EIO error from lookup is good enough to prevert crossing
> > over into the case folded zone by mistake, just as EIO on lookup is
> > enough to deal with the unsupported use case of modifying
> > overlayfs underlying layers with overlay is mounted.
> >
> > BTW, it is not enough to claim that there is no case folding for the
> > entire subtree to allow the mount.
> > For overlayfs to allow d_hash()/d_compare() fs must claim that
> > these implementations are the default implementation in all subtree
> > or at least that all layers share the same implementation.
>

Nevermind. Misread patch 6.

Thanks,
Amir.

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 05:13:37PM +0200, Amir Goldstein wrote:
> On Tue, May 20, 2025 at 4:44 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Tue, May 20, 2025 at 04:33:14PM +0200, Amir Goldstein wrote:
> > > On Tue, May 20, 2025 at 4:12 PM Kent Overstreet
> > > > Amir, you've got two widely used filesystem features that conflict and
> > > > can't be used on the same filesystem.
> > > >
> > > > That's _broken_.
> > >
> > > Correct.
> > >
> > > I am saying that IMO a smaller impact (and less user friendly) fix is more
> > > appropriate way to deal with this problem.
> >
> > Less user friendly is an understatement.
> >
> > Obscure errors that only get reported via overloaded standard error
> > codes is a massive problem today, for _developers_ - have you never had
> > a day of swearing over trying to track down where in a massive subsystem
> > an -EINVAL is coming from?
> >
> > It's even worse for end users that don't know to check the dmesg log.
> >
> > And I support my code, so these would turn into bug reports coming
> > across my desk - no thanks; I already get enough weird shit from other
> > subsystems that I have to look at and at least triage.
> >
> > > > Users hate partitioning just for separate /boot and /home, having to
> > > > partition for different applications is horrible. And since overlay fs
> > > > is used under the hood by docker, and casefolding is used under the hood
> > > > for running Windows applications, this isn't something people can
> > > > predict in advance.
> > >
> > > Right, I am not expecting users to partition by application,
> > > but my question was this:
> > >
> > > When is overlayfs created over a subtree that is only partially case-folded?
> > >
> > > Obviously, docker would create overlayfs on parts of the fs
> > > and smbd/cygwin could create a case folder subtree on another
> > > part of the fs.
> > > I just don't see a common use case when these sections overlap.
> >
> > Today, you cannot user docker and casefolding on _different parts of_
> > the same filesystem.
> >
> > So yees, today users do have to partition by application, or only use
> > one feature or the other.
> >
> 
> Didn't say there was no problem.
> 
> Argued that your fix is a big gun and not worth the added complexity.
> 
> Let's see what Miklos thinks.
> 
> > This isn't about allowing casefolding and overlayfs to fix on the same
> > subtree, that would be a bigger project.
> >
> > > Perhaps I am wrong (please present real world use cases),
> > > but my claim is that this case is not common enough and therefore,
> > > a suboptimal EIO error from lookup is good enough to prevert crossing
> > > over into the case folded zone by mistake, just as EIO on lookup is
> > > enough to deal with the unsupported use case of modifying
> > > overlayfs underlying layers with overlay is mounted.
> > >
> > > BTW, it is not enough to claim that there is no case folding for the
> > > entire subtree to allow the mount.
> > > For overlayfs to allow d_hash()/d_compare() fs must claim that
> > > these implementations are the default implementation in all subtree
> > > or at least that all layers share the same implementation.
> >
> 
> Nevermind. Misread patch 6.

Since you were asking for use cases - docker & related are pretty widely
used for deploying things that are "unwieldy" within the normal packgae
manager ecosystem - and wine is case study #1 in that, where these days
people want to ship a specific version of wine with applications being
emulated (that's been tested with that application).

But wine wants casefolding, so - hapless user deploys docker image where
casefolding is enabled _but only on the subdir that holds Windows data_,
not the whole image.

Docker mounts the image, but then everything explodes when you try to
use it with what look to the user like impenetrable IO errors.

That's a bad day for someone, or more likely a lot of someones.

Re: [PATCH 4/6] fs: dcache locking for exlusion between overlayfs, casefolding

[Al Viro]

On Tue, May 20, 2025 at 01:15:56AM -0400, Kent Overstreet wrote:

> +int d_casefold_enable(struct dentry *dentry, struct d_casefold_enable *e)
> +{
> +	struct dentry *root = dentry->d_sb->s_root;
> +	int ret = 0;
> +
> +	guard(mutex)(&no_casefold_dentries_lock);
> +
> +	for (struct dentry *i = dentry;
> +	     i && i->d_inode->i_flags & S_NO_CASEFOLD;
> +	     i = i != root ? i->d_parent : NULL) {
> +		ret = darray_push(&e->refs, i);
> +		if (ret)
> +			goto err;
> +
> +		ret = no_casefold_dentry_get(i, ref_casefold_enable);

	Beyond being fucking ugly, this is outright broken.  Lose
the timeslice (e.g. on allocation in that thing), and there's
nothing to prevent your 'i' from pointing to freed memory.

Re: [PATCH 4/6] fs: dcache locking for exlusion between overlayfs, casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 04:25:36PM +0100, Al Viro wrote:
> On Tue, May 20, 2025 at 01:15:56AM -0400, Kent Overstreet wrote:
> 
> > +int d_casefold_enable(struct dentry *dentry, struct d_casefold_enable *e)
> > +{
> > +	struct dentry *root = dentry->d_sb->s_root;
> > +	int ret = 0;
> > +
> > +	guard(mutex)(&no_casefold_dentries_lock);
> > +
> > +	for (struct dentry *i = dentry;
> > +	     i && i->d_inode->i_flags & S_NO_CASEFOLD;
> > +	     i = i != root ? i->d_parent : NULL) {
> > +		ret = darray_push(&e->refs, i);
> > +		if (ret)
> > +			goto err;
> > +
> > +		ret = no_casefold_dentry_get(i, ref_casefold_enable);
> 
> 	Beyond being fucking ugly, this is outright broken.  Lose
> the timeslice (e.g. on allocation in that thing), and there's
> nothing to prevent your 'i' from pointing to freed memory.

I was under the impression that dentries couldn't be freed while a child
is pinned, and we have a dget() on the start of the chain.

But no, rename would break that, of course.

Re: [PATCH 0/6] overlayfs + casefolding

[Miklos Szeredi]

On Tue, 20 May 2025 at 17:21, Kent Overstreet <kent.overstreet@linux.dev> wrote:

> Docker mounts the image, but then everything explodes when you try to
> use it with what look to the user like impenetrable IO errors.
>
> That's a bad day for someone, or more likely a lot of someones.

Wouldn't it be docker's responsibility to know that that won't work
with overlayfs?

Any error, whether at startup or during operation is not something the
user will like.

What am I missing?

Thanks,
Miklos

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 06:40:35PM +0200, Miklos Szeredi wrote:
> On Tue, 20 May 2025 at 17:21, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> 
> > Docker mounts the image, but then everything explodes when you try to
> > use it with what look to the user like impenetrable IO errors.
> >
> > That's a bad day for someone, or more likely a lot of someones.
> 
> Wouldn't it be docker's responsibility to know that that won't work
> with overlayfs?

Why would it be?

> Any error, whether at startup or during operation is not something the
> user will like.
> 
> What am I missing?

A _mount_ error due to misconfiguration is expected operation, and we've
built mechanisms for reporting those errors in a way that users can
understand what's going on.

That's not true for normal file accesses post mount.

Re: [PATCH 0/6] overlayfs + casefolding

[John Stoffel]

>>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:

> On Tue, May 20, 2025 at 04:03:27PM +0200, Amir Goldstein wrote:
>> On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
>> <kent.overstreet@linux.dev> wrote:
>> >
>> > On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
>> > > On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
>> > > <kent.overstreet@linux.dev> wrote:
>> > > >
>> > > > On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
>> > > > > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
>> > > > > <kent.overstreet@linux.dev> wrote:
>> > > > > >
>> > > > > > This series allows overlayfs and casefolding to safely be used on the
>> > > > > > same filesystem by providing exclusion to ensure that overlayfs never
>> > > > > > has to deal with casefolded directories.
>> > > > > >
>> > > > > > Currently, overlayfs can't be used _at all_ if a filesystem even
>> > > > > > supports casefolding, which is really nasty for users.
>> > > > > >
>> > > > > > Components:
>> > > > > >
>> > > > > > - filesystem has to track, for each directory, "does any _descendent_
>> > > > > >   have casefolding enabled"
>> > > > > >
>> > > > > > - new inode flag to pass this to VFS layer
>> > > > > >
>> > > > > > - new dcache methods for providing refs for overlayfs, and filesystem
>> > > > > >   methods for safely clearing this flag
>> > > > > >
>> > > > > > - new superblock flag for indicating to overlayfs & dcache "filesystem
>> > > > > >   supports casefolding, it's safe to use provided new dcache methods are
>> > > > > >   used"
>> > > > > >
>> > > > >
>> > > > > I don't think that this is really needed.
>> > > > >
>> > > > > Too bad you did not ask before going through the trouble of this implementation.
>> > > > >
>> > > > > I think it is enough for overlayfs to know the THIS directory has no
>> > > > > casefolding.
>> > > >
>> > > > overlayfs works on trees, not directories...
>> > >
>> > > I know how overlayfs works...
>> > >
>> > > I've explained why I don't think that sanitizing the entire tree is needed
>> > > for creating overlayfs over a filesystem that may enable casefolding
>> > > on some of its directories.
>> >
>> > So, you want to move error checking from mount time, where we _just_
>> > did a massive API rework so that we can return errors in a way that
>> > users will actually see them - to open/lookup, where all we have are a
>> > small fixed set of error codes?
>> 
>> That's one way of putting it.
>> 
>> Please explain the use case.
>> 
>> When is overlayfs created over a subtree that is only partially case folded?
>> Is that really so common that a mount time error justifies all the vfs
>> infrastructure involved?

> Amir, you've got two widely used filesystem features that conflict and
> can't be used on the same filesystem.

Wait, what?  How many people use casefolding, on a per-directory
basis?  It's stupid.  Unix/Linux has used case-sensitive filesystems
for years.  Yes, linux supports other OSes which did do casefolding,
but yikes... per-directory support is just insane.  It should be
per-filesystem only at BEST.  

> That's _broken_.

So?  what about my cross mounting of VMS filesystems with "foo.txt;3"
version control so I can go back to previous versions?  Why can't I do
that from my Linux systems that's mounting that VMS image?   

Just because it's done doesn't mean it's not dumb.  

> Users hate partitioning just for separate /boot and /home, having to
> partition for different applications is horrible. And since overlay
> fs is used under the hood by docker, and casefolding is used under
> the hood for running Windows applications, this isn't something
> people can predict in advance.

Sure I can, I don't run windows applications to screw casefolding.
:-)

And I personally LIKE having a seperate /boot and /home, because it
gives isolation.  The world is not just single user laptops with
everything all on one disk or spread across a couple of disks using
LVM or RAID or all of the above.  

I also don't see any updates for the XFS tests, or any other
filesystem tests, that actually checks and confirms this decidedly
obtuse and dumb to implement idea.  


John

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 02:49:16PM -0400, John Stoffel wrote:
> >>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:
> 
> > On Tue, May 20, 2025 at 04:03:27PM +0200, Amir Goldstein wrote:
> >> On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
> >> <kent.overstreet@linux.dev> wrote:
> >> >
> >> > On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
> >> > > On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
> >> > > <kent.overstreet@linux.dev> wrote:
> >> > > >
> >> > > > On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
> >> > > > > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
> >> > > > > <kent.overstreet@linux.dev> wrote:
> >> > > > > >
> >> > > > > > This series allows overlayfs and casefolding to safely be used on the
> >> > > > > > same filesystem by providing exclusion to ensure that overlayfs never
> >> > > > > > has to deal with casefolded directories.
> >> > > > > >
> >> > > > > > Currently, overlayfs can't be used _at all_ if a filesystem even
> >> > > > > > supports casefolding, which is really nasty for users.
> >> > > > > >
> >> > > > > > Components:
> >> > > > > >
> >> > > > > > - filesystem has to track, for each directory, "does any _descendent_
> >> > > > > >   have casefolding enabled"
> >> > > > > >
> >> > > > > > - new inode flag to pass this to VFS layer
> >> > > > > >
> >> > > > > > - new dcache methods for providing refs for overlayfs, and filesystem
> >> > > > > >   methods for safely clearing this flag
> >> > > > > >
> >> > > > > > - new superblock flag for indicating to overlayfs & dcache "filesystem
> >> > > > > >   supports casefolding, it's safe to use provided new dcache methods are
> >> > > > > >   used"
> >> > > > > >
> >> > > > >
> >> > > > > I don't think that this is really needed.
> >> > > > >
> >> > > > > Too bad you did not ask before going through the trouble of this implementation.
> >> > > > >
> >> > > > > I think it is enough for overlayfs to know the THIS directory has no
> >> > > > > casefolding.
> >> > > >
> >> > > > overlayfs works on trees, not directories...
> >> > >
> >> > > I know how overlayfs works...
> >> > >
> >> > > I've explained why I don't think that sanitizing the entire tree is needed
> >> > > for creating overlayfs over a filesystem that may enable casefolding
> >> > > on some of its directories.
> >> >
> >> > So, you want to move error checking from mount time, where we _just_
> >> > did a massive API rework so that we can return errors in a way that
> >> > users will actually see them - to open/lookup, where all we have are a
> >> > small fixed set of error codes?
> >> 
> >> That's one way of putting it.
> >> 
> >> Please explain the use case.
> >> 
> >> When is overlayfs created over a subtree that is only partially case folded?
> >> Is that really so common that a mount time error justifies all the vfs
> >> infrastructure involved?
> 
> > Amir, you've got two widely used filesystem features that conflict and
> > can't be used on the same filesystem.
> 
> Wait, what?  How many people use casefolding, on a per-directory
> basis?  It's stupid.  Unix/Linux has used case-sensitive filesystems
> for years.  Yes, linux supports other OSes which did do casefolding,
> but yikes... per-directory support is just insane.  It should be
> per-filesystem only at BEST.  

Quite a lot.

You may not realize this, but Valve has, quietly, behind the scenes,
been intelligently funding a ton of Linux work with an eye towards not
just gaming, but improving Linus on the desktop. And they've been
deploying it too, you can buy a Steam deck today.

And a significant fraction of desktop users like to play games - we're
not just all work. Windows ports need casefolding - alternatives have
been discussed and they're non viable.

(I fondly remember the days when I had time for such things).

Samba fileservers are a thing, too.

And for all us desktop/workstation users, not being able to use the same
filesystem for wine and docker is the kind of jankiness that makes
people say "maybe Linux isn't ready for the desktop after all".

Put aside your feelings on casefolding - this is about basic attention
to detail.

> > That's _broken_.
> 
> So?  what about my cross mounting of VMS filesystems with "foo.txt;3"
> version control so I can go back to previous versions?  Why can't I do
> that from my Linux systems that's mounting that VMS image?   
> 
> Just because it's done doesn't mean it's not dumb.  
> 
> > Users hate partitioning just for separate /boot and /home, having to
> > partition for different applications is horrible. And since overlay
> > fs is used under the hood by docker, and casefolding is used under
> > the hood for running Windows applications, this isn't something
> > people can predict in advance.
> 
> Sure I can, I don't run windows applications to screw casefolding.
> :-)
> 
> And I personally LIKE having a seperate /boot and /home, because it
> gives isolation.  The world is not just single user laptops with
> everything all on one disk or spread across a couple of disks using
> LVM or RAID or all of the above.  
> 
> I also don't see any updates for the XFS tests, or any other
> filesystem tests, that actually checks and confirms this decidedly
> obtuse and dumb to implement idea.  

Well, you certainly are making your personal feelings known :)

But for me, as the engineer designing and implementing this stuff, I
don't let my personal feelings dictate.

That's not my place.

My job is to write code that works, works reliably, solves real
problems, and lets users do the things they want with their machines.

All this drama over casefolding has been pure distraction, and I would
appreciate if you and everyone else could tone it down now.

Re: [PATCH 0/6] overlayfs + casefolding

[Malte Schröder]

On 20/05/2025 20:49, John Stoffel wrote:
>>>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:
>> On Tue, May 20, 2025 at 04:03:27PM +0200, Amir Goldstein wrote:
>>> On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
>>> <kent.overstreet@linux.dev> wrote:
>>>> On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
>>>>> On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
>>>>> <kent.overstreet@linux.dev> wrote:
>>>>>> On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
>>>>>>> On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
>>>>>>> <kent.overstreet@linux.dev> wrote:
>>>>>>>> This series allows overlayfs and casefolding to safely be used on the
>>>>>>>> same filesystem by providing exclusion to ensure that overlayfs never
>>>>>>>> has to deal with casefolded directories.
>>>>>>>>
>>>>>>>> Currently, overlayfs can't be used _at all_ if a filesystem even
>>>>>>>> supports casefolding, which is really nasty for users.
>>>>>>>>
>>>>>>>> Components:
>>>>>>>>
>>>>>>>> - filesystem has to track, for each directory, "does any _descendent_
>>>>>>>>   have casefolding enabled"
>>>>>>>>
>>>>>>>> - new inode flag to pass this to VFS layer
>>>>>>>>
>>>>>>>> - new dcache methods for providing refs for overlayfs, and filesystem
>>>>>>>>   methods for safely clearing this flag
>>>>>>>>
>>>>>>>> - new superblock flag for indicating to overlayfs & dcache "filesystem
>>>>>>>>   supports casefolding, it's safe to use provided new dcache methods are
>>>>>>>>   used"
>>>>>>>>
>>>>>>> I don't think that this is really needed.
>>>>>>>
>>>>>>> Too bad you did not ask before going through the trouble of this implementation.
>>>>>>>
>>>>>>> I think it is enough for overlayfs to know the THIS directory has no
>>>>>>> casefolding.
>>>>>> overlayfs works on trees, not directories...
>>>>> I know how overlayfs works...
>>>>>
>>>>> I've explained why I don't think that sanitizing the entire tree is needed
>>>>> for creating overlayfs over a filesystem that may enable casefolding
>>>>> on some of its directories.
>>>> So, you want to move error checking from mount time, where we _just_
>>>> did a massive API rework so that we can return errors in a way that
>>>> users will actually see them - to open/lookup, where all we have are a
>>>> small fixed set of error codes?
>>> That's one way of putting it.
>>>
>>> Please explain the use case.
>>>
>>> When is overlayfs created over a subtree that is only partially case folded?
>>> Is that really so common that a mount time error justifies all the vfs
>>> infrastructure involved?
>> Amir, you've got two widely used filesystem features that conflict and
>> can't be used on the same filesystem.
> Wait, what?  How many people use casefolding, on a per-directory
> basis?  It's stupid.  Unix/Linux has used case-sensitive filesystems
> for years.  Yes, linux supports other OSes which did do casefolding,
> but yikes... per-directory support is just insane.  It should be
> per-filesystem only at BEST.  
>
>> That's _broken_.
> So?  what about my cross mounting of VMS filesystems with "foo.txt;3"
> version control so I can go back to previous versions?  Why can't I do
> that from my Linux systems that's mounting that VMS image?   
>
> Just because it's done doesn't mean it's not dumb.  
>
>> Users hate partitioning just for separate /boot and /home, having to
>> partition for different applications is horrible. And since overlay
>> fs is used under the hood by docker, and casefolding is used under
>> the hood for running Windows applications, this isn't something
>> people can predict in advance.
> Sure I can, I don't run windows applications to screw casefolding.
> :-)
>
> And I personally LIKE having a seperate /boot and /home, because it
> gives isolation.  The world is not just single user laptops with
> everything all on one disk or spread across a couple of disks using
> LVM or RAID or all of the above.  
>
> I also don't see any updates for the XFS tests, or any other
> filesystem tests, that actually checks and confirms this decidedly
> obtuse and dumb to implement idea.  
>
>
> John
>
Hi there,

would you partition different subdirs of your /home? So there is
.local/share/containers where users put their container-stuff (at least
podman does). Then there is .wine where case-folding-craziness lives.
And then there is the mess that is Steam, which does all kinds of
containery case-foldy stuff. As much as I would like to keep these
things apart, it is not feasible. Not for me as a "power user", and
certainly far out of reach for average Joe user.

Just my 2 ct, greets

/Malte

Re: [PATCH 0/6] overlayfs + casefolding

[Christopher Snowhill]

On Wed May 21, 2025 at 4:26 AM PDT, Malte Schröder wrote:
> On 20/05/2025 20:49, John Stoffel wrote:
>>>>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:
>>> On Tue, May 20, 2025 at 04:03:27PM +0200, Amir Goldstein wrote:
>>>> On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
>>>> <kent.overstreet@linux.dev> wrote:
>>>>> On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
>>>>>> On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
>>>>>> <kent.overstreet@linux.dev> wrote:
>>>>>>> On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
>>>>>>>> On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
>>>>>>>> <kent.overstreet@linux.dev> wrote:
>>>>>>>>> This series allows overlayfs and casefolding to safely be used on the
>>>>>>>>> same filesystem by providing exclusion to ensure that overlayfs never
>>>>>>>>> has to deal with casefolded directories.
>>>>>>>>>
>>>>>>>>> Currently, overlayfs can't be used _at all_ if a filesystem even
>>>>>>>>> supports casefolding, which is really nasty for users.
>>>>>>>>>
>>>>>>>>> Components:
>>>>>>>>>
>>>>>>>>> - filesystem has to track, for each directory, "does any _descendent_
>>>>>>>>>   have casefolding enabled"
>>>>>>>>>
>>>>>>>>> - new inode flag to pass this to VFS layer
>>>>>>>>>
>>>>>>>>> - new dcache methods for providing refs for overlayfs, and filesystem
>>>>>>>>>   methods for safely clearing this flag
>>>>>>>>>
>>>>>>>>> - new superblock flag for indicating to overlayfs & dcache "filesystem
>>>>>>>>>   supports casefolding, it's safe to use provided new dcache methods are
>>>>>>>>>   used"
>>>>>>>>>
>>>>>>>> I don't think that this is really needed.
>>>>>>>>
>>>>>>>> Too bad you did not ask before going through the trouble of this implementation.
>>>>>>>>
>>>>>>>> I think it is enough for overlayfs to know the THIS directory has no
>>>>>>>> casefolding.
>>>>>>> overlayfs works on trees, not directories...
>>>>>> I know how overlayfs works...
>>>>>>
>>>>>> I've explained why I don't think that sanitizing the entire tree is needed
>>>>>> for creating overlayfs over a filesystem that may enable casefolding
>>>>>> on some of its directories.
>>>>> So, you want to move error checking from mount time, where we _just_
>>>>> did a massive API rework so that we can return errors in a way that
>>>>> users will actually see them - to open/lookup, where all we have are a
>>>>> small fixed set of error codes?
>>>> That's one way of putting it.
>>>>
>>>> Please explain the use case.
>>>>
>>>> When is overlayfs created over a subtree that is only partially case folded?
>>>> Is that really so common that a mount time error justifies all the vfs
>>>> infrastructure involved?
>>> Amir, you've got two widely used filesystem features that conflict and
>>> can't be used on the same filesystem.
>> Wait, what?  How many people use casefolding, on a per-directory
>> basis?  It's stupid.  Unix/Linux has used case-sensitive filesystems
>> for years.  Yes, linux supports other OSes which did do casefolding,
>> but yikes... per-directory support is just insane.  It should be
>> per-filesystem only at BEST.  
>>
>>> That's _broken_.
>> So?  what about my cross mounting of VMS filesystems with "foo.txt;3"
>> version control so I can go back to previous versions?  Why can't I do
>> that from my Linux systems that's mounting that VMS image?   
>>
>> Just because it's done doesn't mean it's not dumb.  
>>
>>> Users hate partitioning just for separate /boot and /home, having to
>>> partition for different applications is horrible. And since overlay
>>> fs is used under the hood by docker, and casefolding is used under
>>> the hood for running Windows applications, this isn't something
>>> people can predict in advance.
>> Sure I can, I don't run windows applications to screw casefolding.
>> :-)
>>
>> And I personally LIKE having a seperate /boot and /home, because it
>> gives isolation.  The world is not just single user laptops with
>> everything all on one disk or spread across a couple of disks using
>> LVM or RAID or all of the above.  
>>
>> I also don't see any updates for the XFS tests, or any other
>> filesystem tests, that actually checks and confirms this decidedly
>> obtuse and dumb to implement idea.  
>>
>>
>> John
>>
> Hi there,
>
> would you partition different subdirs of your /home? So there is
> .local/share/containers where users put their container-stuff (at least
> podman does). Then there is .wine where case-folding-craziness lives.
> And then there is the mess that is Steam, which does all kinds of
> containery case-foldy stuff. As much as I would like to keep these
> things apart, it is not feasible. Not for me as a "power user", and
> certainly far out of reach for average Joe user.
>
> Just my 2 ct, greets

"But just disable it globally" How about no. Sure, ext4 has that flag,
but bcachefs by design does not. And a change that already made it into
6.15 has made it trip overlayfs as it is now, unconditionally. The
purpose of this new implementation is to make it work, and satisfy the
condition that overlayfs is guaranteed no casefolding in its tree, and
that nobody may create a new folder inside that tree and suddenly turn
on casefolding on it.

And this change also makes it possible to use it with ext4 with the
global casefolding flag enabled. It shouldn't be necessary to have a
global killswitch, these features should be able to live together on the
same filesystem as long as they're not touching each other, and aren't
allowed to touch each other.

>
> /Malte

Re: [PATCH 0/6] overlayfs + casefolding

[John Stoffel]

>>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:

> On Tue, May 20, 2025 at 02:49:16PM -0400, John Stoffel wrote:
>> >>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:
>> 
>> > On Tue, May 20, 2025 at 04:03:27PM +0200, Amir Goldstein wrote:
>> >> On Tue, May 20, 2025 at 2:43 PM Kent Overstreet
>> >> <kent.overstreet@linux.dev> wrote:
>> >> >
>> >> > On Tue, May 20, 2025 at 02:40:07PM +0200, Amir Goldstein wrote:
>> >> > > On Tue, May 20, 2025 at 2:25 PM Kent Overstreet
>> >> > > <kent.overstreet@linux.dev> wrote:
>> >> > > >
>> >> > > > On Tue, May 20, 2025 at 10:05:14AM +0200, Amir Goldstein wrote:
>> >> > > > > On Tue, May 20, 2025 at 7:16 AM Kent Overstreet
>> >> > > > > <kent.overstreet@linux.dev> wrote:
>> >> > > > > >
>> >> > > > > > This series allows overlayfs and casefolding to safely be used on the
>> >> > > > > > same filesystem by providing exclusion to ensure that overlayfs never
>> >> > > > > > has to deal with casefolded directories.
>> >> > > > > >
>> >> > > > > > Currently, overlayfs can't be used _at all_ if a filesystem even
>> >> > > > > > supports casefolding, which is really nasty for users.
>> >> > > > > >
>> >> > > > > > Components:
>> >> > > > > >
>> >> > > > > > - filesystem has to track, for each directory, "does any _descendent_
>> >> > > > > >   have casefolding enabled"
>> >> > > > > >
>> >> > > > > > - new inode flag to pass this to VFS layer
>> >> > > > > >
>> >> > > > > > - new dcache methods for providing refs for overlayfs, and filesystem
>> >> > > > > >   methods for safely clearing this flag
>> >> > > > > >
>> >> > > > > > - new superblock flag for indicating to overlayfs & dcache "filesystem
>> >> > > > > >   supports casefolding, it's safe to use provided new dcache methods are
>> >> > > > > >   used"
>> >> > > > > >
>> >> > > > >
>> >> > > > > I don't think that this is really needed.
>> >> > > > >
>> >> > > > > Too bad you did not ask before going through the trouble of this implementation.
>> >> > > > >
>> >> > > > > I think it is enough for overlayfs to know the THIS directory has no
>> >> > > > > casefolding.
>> >> > > >
>> >> > > > overlayfs works on trees, not directories...
>> >> > >
>> >> > > I know how overlayfs works...
>> >> > >
>> >> > > I've explained why I don't think that sanitizing the entire tree is needed
>> >> > > for creating overlayfs over a filesystem that may enable casefolding
>> >> > > on some of its directories.
>> >> >
>> >> > So, you want to move error checking from mount time, where we _just_
>> >> > did a massive API rework so that we can return errors in a way that
>> >> > users will actually see them - to open/lookup, where all we have are a
>> >> > small fixed set of error codes?
>> >> 
>> >> That's one way of putting it.
>> >> 
>> >> Please explain the use case.
>> >> 
>> >> When is overlayfs created over a subtree that is only partially case folded?
>> >> Is that really so common that a mount time error justifies all the vfs
>> >> infrastructure involved?
>> 
>> > Amir, you've got two widely used filesystem features that conflict and
>> > can't be used on the same filesystem.
>> 
>> Wait, what?  How many people use casefolding, on a per-directory
>> basis?  It's stupid.  Unix/Linux has used case-sensitive filesystems
>> for years.  Yes, linux supports other OSes which did do casefolding,
>> but yikes... per-directory support is just insane.  It should be
>> per-filesystem only at BEST.  

> Quite a lot.

As I'm learning!  :-)    And sorry for the late reply... life got in
the way...

> You may not realize this, but Valve has, quietly, behind the scenes,
> been intelligently funding a ton of Linux work with an eye towards
> not just gaming, but improving Linus on the desktop. And they've
> been deploying it too, you can buy a Steam deck today.

So... this arguement that you're improving the desktop by doing
case-folding on filesystems smells bogus to me.  I think it's more
correct to say "Valve wants windows application support on top of
linux, and since windows does case-folding, linux should too."

Which is more an arguement for supporting legacy backwards
implementations.  But for the future?  God, why? 

> And a significant fraction of desktop users like to play games - we're
> not just all work. Windows ports need casefolding - alternatives have
> been discussed and they're non viable.

> (I fondly remember the days when I had time for such things).

Heh heh.  I'm not a Windows gamer at all, which is why I haven't run
into this at all.  

> Samba fileservers are a thing, too.

Sure.  And I understand that Windows (even Windows 11!) is still does
casefolding by default.  Sigh... which sucks.  

> And for all us desktop/workstation users, not being able to use the same
> filesystem for wine and docker is the kind of jankiness that makes
> people say "maybe Linux isn't ready for the desktop after all".

I dunno... I just wonder why overlayfs (or some other level) can't
just hide this and leave the lower levels alone.  Yes, when you have
'foo' and 'Foo' on Linux, how do you handle it?  It's painful.

> Put aside your feelings on casefolding - this is about basic attention
> to detail.

Heh.  I'm getting an education here.  And disliking all the hoops
filesystems have to go through to support this (IMHO) dumb feature of
windows.  

>> > That's _broken_.
>> 
>> So?  what about my cross mounting of VMS filesystems with "foo.txt;3"
>> version control so I can go back to previous versions?  Why can't I do
>> that from my Linux systems that's mounting that VMS image?   
>> 
>> Just because it's done doesn't mean it's not dumb.  
>> 
>> > Users hate partitioning just for separate /boot and /home, having to
>> > partition for different applications is horrible. And since overlay
>> > fs is used under the hood by docker, and casefolding is used under
>> > the hood for running Windows applications, this isn't something
>> > people can predict in advance.
>> 
>> Sure I can, I don't run windows applications to screw casefolding.
>> :-)
>> 
>> And I personally LIKE having a seperate /boot and /home, because it
>> gives isolation.  The world is not just single user laptops with
>> everything all on one disk or spread across a couple of disks using
>> LVM or RAID or all of the above.  
>> 
>> I also don't see any updates for the XFS tests, or any other
>> filesystem tests, that actually checks and confirms this decidedly
>> obtuse and dumb to implement idea.  

> Well, you certainly are making your personal feelings known :)

> But for me, as the engineer designing and implementing this stuff, I
> don't let my personal feelings dictate.

> That's not my place.

> My job is to write code that works, works reliably, solves real
> problems, and lets users do the things they want with their machines.

> All this drama over casefolding has been pure distraction, and I would
> appreciate if you and everyone else could tone it down now.

[PATCH v2] fs: dcache exlusion between overlayfs, casefolding

[Kent Overstreet]

Allow casefolding to be used on the same filesystem as overlayfs.

Overlayfs cannot handle a directory that has been casefolding, so this
implements locking between enabling casefolding and overlayfs.

The underlying filesystem mus also track, for each directory, whether
casefolding has been enabled on any subdirectory, and provide this
information to the VFS with the S_NO_CASEFOLD flag.

Since we can't inflate struct dentry for this, add a separate rhashtable
that functions as a per-dentry "casefolding disabled" ref.

New helpers, for overlayfs:
- d_casefold_disabled_put()
- d_casefold_disabled_get()

These acquire and release refs that check that S_NO_CASEFOLD is set on a
directory, and prevent it from being cleared while the ref exists.

For the underlying filesystem:

- d_casefolding_enable()
- d_casefolding_enable_commit()

This is for the underlying filesystem that implements casefolding, in
settar and rename. d_casefolding_enable() checks for conflicting refs
held by overlayfs and acquires refs while the rename or setatr is being
done, d_casefolding_enable_commit() releases those refs and clears the
S_NO_CASEFOLD flag.

Implementation -

We can't inflate struct dentry for this, so references are tracked in a
separate rhashtable.

To guard against races with rename(), d_casefolding_enable() must take
refs (which in turn call dget()) on all dentries with inodes that need
S_NO_CASEFOLD flipped; these dget() refs are released by commit().

d_casefold_disabled_get/put take dget() refs for the duration of the
union mount, but this is unchanged because unionfs already holds a path
ref for the duration of the mount.

Changes since v1:

- Fix "walk up the tree" synchronization

  s_vfs_rename_mutex is required: we can't allow an operation enabling
  casefolding (via fileattr_set) to race with a rename moving it into a
  tree exported by overlayfs.

  d_casefold_enable() has a new @rename parameter: if set, we check that
  s_vfs_rename_mutex is held, if not (called from fileattr_set), we
  acquire it and release it in d_casefold_enable_commit().

  Also, use dget_parent().

- no_casefold_dentries_lock -> sb->s_casefold_enable_lock

- d_casefold_enable() now bails out early if called on non-directory, or
  if S_NO_CASEFOLD is not set.

Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
---
 fs/dcache.c            | 266 +++++++++++++++++++++++++++++++++++++++++
 fs/super.c             |   1 +
 include/linux/dcache.h |  12 ++
 include/linux/fs.h     |   4 +
 4 files changed, 283 insertions(+)

diff --git a/fs/dcache.c b/fs/dcache.c
index bd5aa136153a..d29faa6f9ec8 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -32,6 +32,8 @@
 #include <linux/bit_spinlock.h>
 #include <linux/rculist_bl.h>
 #include <linux/list_lru.h>
+#include <linux/rhashtable.h>
+#include <linux/darray.h>
 #include "internal.h"
 #include "mount.h"
 
@@ -3141,6 +3143,266 @@ ino_t d_parent_ino(struct dentry *dentry)
 }
 EXPORT_SYMBOL(d_parent_ino);
 
+static struct rhashtable no_casefold_dentries;
+
+enum no_casefold_dentry_ref {
+	ref_casefold_disable,
+	ref_casefold_enable,
+};
+
+struct no_casefold_dentry {
+	struct rhash_head	hash;
+	struct dentry		*dentry;
+	unsigned long		ref[2];
+};
+
+static const struct rhashtable_params no_casefold_dentries_params = {
+	.head_offset		= offsetof(struct no_casefold_dentry, hash),
+	.key_offset		= offsetof(struct no_casefold_dentry, dentry),
+	.key_len		= sizeof(struct dentry *),
+	.automatic_shrinking	= true,
+};
+
+static int no_casefold_dentry_get(struct dentry *dentry,
+				  enum no_casefold_dentry_ref ref)
+{
+	struct no_casefold_dentry *n =
+		rhashtable_lookup_fast(&no_casefold_dentries,
+				       &dentry,
+				       no_casefold_dentries_params);
+	if (n) {
+		if (n->ref[!ref])
+			return -EINVAL;
+
+		n->ref[ref]++;
+		return 0;
+	}
+
+	n = kzalloc(sizeof(*n), GFP_KERNEL);
+	if (!n)
+		return -ENOMEM;
+
+	n->dentry = dget(dentry);
+	n->ref[ref]++;
+
+	int ret = rhashtable_lookup_insert_fast(&no_casefold_dentries,
+				&n->hash, no_casefold_dentries_params);
+	if (WARN_ON(ret)) {
+		kfree(n);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void no_casefold_dentry_put(struct dentry *dentry,
+				   enum no_casefold_dentry_ref ref)
+{
+	struct no_casefold_dentry *n =
+		rhashtable_lookup_fast(&no_casefold_dentries,
+				       &dentry,
+				       no_casefold_dentries_params);
+	if (WARN_ON(!n))
+		return;
+
+	if (--n->ref[ref])
+		return;
+
+	dput(n->dentry);
+	int ret = rhashtable_remove_fast(&no_casefold_dentries,
+					 &n->hash, no_casefold_dentries_params);
+	WARN_ON(ret);
+}
+
+/**
+ * d_casefold_disabled_put - drop a "casefold disabled" ref
+ *
+ * Only for overlayfs.
+ */
+void d_casefold_disabled_put(struct dentry *dentry)
+{
+	struct super_block *sb = dentry->d_sb;
+
+	if (!(sb->s_flags & SB_CASEFOLD))
+		return;
+
+	guard(mutex)(&sb->s_casefold_enable_lock);
+	no_casefold_dentry_put(dentry, ref_casefold_disable);
+}
+EXPORT_SYMBOL_GPL(d_casefold_disabled_put);
+
+/**
+ * d_casefold_disabled_get - attempt to disable casefold on a tree
+ *
+ * Only for overlayfs.
+ *
+ * Returns -EINVAL if casefolding is in use on any subdirectory; this must be
+ * tracked by the filesystem.
+ *
+ * On success, returns with a reference held that must be released by
+ * d_casefold_disabled_put(); this ref blocks casefold from being enabled
+ * by d_casefold_enable().
+ */
+int d_casefold_disabled_get(struct dentry *dentry)
+{
+	struct super_block *sb = dentry->d_sb;
+
+	if (!(sb->s_flags & SB_CASEFOLD))
+		return 0;
+
+	guard(mutex)(&sb->s_casefold_enable_lock);
+
+	if (!(dentry->d_inode->i_flags & S_NO_CASEFOLD))
+		return -EINVAL;
+
+	return no_casefold_dentry_get(dentry, ref_casefold_disable);
+}
+EXPORT_SYMBOL_GPL(d_casefold_disabled_get);
+
+/* Crabwalk: releases @dentry after getting ref on parent */
+static struct dentry *dget_parent_this_sb(struct dentry *dentry)
+{
+	struct dentry *parent = dentry != dentry->d_sb->s_root
+		? dget_parent(dentry)
+		: NULL;
+	dput(dentry);
+	return parent;
+}
+
+/**
+ * d_casefold_enable - check if casefolding may be enabled on a dentry
+ *
+ * @dentry:	dentry to enable casefolding on
+ * @e:		state object, released by d_casefold_enable_commit()
+ * @rename:	Are we in the rename path?
+ *		If so, we expect s_vfs_rename_mutex to be held, if not (called
+ *		from setflags), we aquire it if necessary, and release in
+ *		commit.
+ *
+ * The rename mutex is required because we're operating on a whole path,
+ * potentially up to the filesystem root, and we need it to be stable until
+ * commit (i.e. we don't want to be renamed into a tree overlayfs is exporting
+ * after we've returned success).
+ *
+ * For rename, this should only be called for cross-directory rename.
+ * S_NO_CASEFOLD doesn't need to change on rename within a directory, and
+ * s_vfs_rename_mutex won't be held on non cross-directory rename.
+ *
+ * Returns -EINVAL if casefolding has been disabled on any parent directory (by
+ * overlayfs).
+ *
+ * On success, the d_casefold_enable object must be committed with
+ * d_casefold_enable_commit(), after the filesystem has updated its internal
+ * state.
+ *
+ * Commit will clear S_NO_CASEFOLD on all inodes up to the filesystem root,
+ * informing overlayfs that this tree has casefolding enabled somewhere in it.
+ */
+int d_casefold_enable(struct dentry *dentry, struct d_casefold_enable *e,
+		      bool rename)
+{
+	int ret = 0;
+
+	memset(e, 0, sizeof(*e));
+	e->sb = dentry->d_sb;
+
+	if (!(e->sb->s_flags & SB_CASEFOLD))
+		return 0;
+
+	if (!S_ISDIR(dentry->d_inode->i_mode))
+		return 0;
+
+	/*
+	 * On rename, we're passed the dentry being renamed (the filesystem is
+	 * not passed the dentry of the directory we're renaming to), but it's
+	 * the parent that may need to have S_NO_CASEFOLD cleared:
+	 */
+	dentry = rename
+		? dget_parent(dentry)
+		: dget(dentry);
+
+	if (!(dentry->d_inode->i_flags & S_NO_CASEFOLD)) {
+		dput(dentry);
+		return 0;
+	}
+
+	if (rename) {
+		lockdep_assert_held(&e->sb->s_vfs_rename_mutex);
+	} else {
+		mutex_lock(&e->sb->s_vfs_rename_mutex);
+		e->rename_mutex_held = true;
+	}
+
+	guard(mutex)(&e->sb->s_casefold_enable_lock);
+
+	for (struct dentry *i = dentry; i; i = dget_parent_this_sb(i)) {
+		if (!(i->d_inode->i_flags & S_NO_CASEFOLD)) {
+			dput(i);
+			break;
+		}
+
+		ret = darray_push(&e->refs, i);
+		if (ret) {
+			dput(i);
+			goto err;
+		}
+
+		ret = no_casefold_dentry_get(i, ref_casefold_enable);
+		if (ret) {
+			dput(i);
+			--e->refs.nr;
+			goto err;
+		}
+	}
+
+	return 0;
+err:
+	darray_for_each(e->refs, i)
+		no_casefold_dentry_put(*i, ref_casefold_enable);
+	darray_exit(&e->refs);
+
+	if (e->rename_mutex_held)
+		mutex_unlock(&e->sb->s_vfs_rename_mutex);
+	e->rename_mutex_held = false;
+	return ret;
+}
+EXPORT_SYMBOL_GPL(d_casefold_enable);
+
+/**
+ * d_casefold_enable_commit - finish operation started by d_casefold_enable()
+ *
+ * @e:		state object, started by d_casefold_enable_commit()
+ * @ret:	Success or failure of the operation, from the filesystem
+ *
+ * On success (@ret == 0), clear S_NO_CASEFOLD on all inodes up to the
+ * filesystem root that have it set, which d_casefold_enable() previously took
+ * references to.
+ */
+void d_casefold_enable_commit(struct d_casefold_enable *e, int ret)
+{
+	if (e->refs.nr) {
+		guard(mutex)(&e->sb->s_casefold_enable_lock);
+
+		darray_for_each(e->refs, i) {
+			if (!ret) {
+				struct inode *inode = (*i)->d_inode;
+
+				spin_lock(&inode->i_lock);
+				inode->i_flags &= ~S_NO_CASEFOLD;
+				spin_unlock(&inode->i_lock);
+			}
+
+			no_casefold_dentry_put(*i, ref_casefold_enable);
+		}
+		darray_exit(&e->refs);
+	}
+
+	if (e->rename_mutex_held)
+		mutex_unlock(&e->sb->s_vfs_rename_mutex);
+	e->rename_mutex_held = false;
+}
+EXPORT_SYMBOL_GPL(d_casefold_enable_commit);
+
 static __initdata unsigned long dhash_entries;
 static int __init set_dhash_entries(char *str)
 {
@@ -3186,6 +3448,10 @@ static void __init dcache_init(void)
 		SLAB_RECLAIM_ACCOUNT|SLAB_PANIC|SLAB_ACCOUNT,
 		d_shortname.string);
 
+	int ret = rhashtable_init(&no_casefold_dentries, &no_casefold_dentries_params);
+	if (ret)
+		panic("error initializing no_casefold_dentries: %i\n", ret);
+
 	/* Hash may have been set up in dcache_init_early */
 	if (!hashdist)
 		return;
diff --git a/fs/super.c b/fs/super.c
index 97a17f9d9023..2ba3446f09d6 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -368,6 +368,7 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags,
 	atomic_set(&s->s_active, 1);
 	mutex_init(&s->s_vfs_rename_mutex);
 	lockdep_set_class(&s->s_vfs_rename_mutex, &type->s_vfs_rename_key);
+	mutex_init(&s->s_casefold_enable_lock);
 	init_rwsem(&s->s_dquot.dqio_sem);
 	s->s_maxbytes = MAX_NON_LFS;
 	s->s_op = &default_op;
diff --git a/include/linux/dcache.h b/include/linux/dcache.h
index e9f07e37dd6f..fb78de44a929 100644
--- a/include/linux/dcache.h
+++ b/include/linux/dcache.h
@@ -3,6 +3,7 @@
 #define __LINUX_DCACHE_H
 
 #include <linux/atomic.h>
+#include <linux/darray_types.h>
 #include <linux/list.h>
 #include <linux/math.h>
 #include <linux/rculist.h>
@@ -604,4 +605,15 @@ static inline struct dentry *d_next_sibling(const struct dentry *dentry)
 	return hlist_entry_safe(dentry->d_sib.next, struct dentry, d_sib);
 }
 
+void d_casefold_disabled_put(struct dentry *dentry);
+int d_casefold_disabled_get(struct dentry *dentry);
+
+struct d_casefold_enable {
+	DARRAY(struct dentry *)	refs;
+	struct super_block	*sb;
+	bool			rename_mutex_held;
+};
+int d_casefold_enable(struct dentry *dentry, struct d_casefold_enable *e, bool);
+void d_casefold_enable_commit(struct d_casefold_enable *e, int ret);
+
 #endif	/* __LINUX_DCACHE_H */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index ba942cd2fea1..7ee0cd7bc15b 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -48,6 +48,7 @@
 #include <linux/rw_hint.h>
 #include <linux/file_ref.h>
 #include <linux/unicode.h>
+#include <linux/rhashtable-types.h>
 
 #include <asm/byteorder.h>
 #include <uapi/linux/fs.h>
@@ -1397,6 +1398,7 @@ struct super_block {
 	 * even looking at it. You had been warned.
 	 */
 	struct mutex s_vfs_rename_mutex;	/* Kludge */
+	struct mutex s_casefold_enable_lock;
 
 	/*
 	 * Filesystem subtype.  If non-empty the filesystem type field
@@ -1440,6 +1442,7 @@ struct super_block {
 
 	struct mutex		s_sync_lock;	/* sync serialisation lock */
 
+
 	/*
 	 * Indicates how deep in a filesystem stack this SB is
 	 */
@@ -2345,6 +2348,7 @@ struct super_operations {
 #define S_CASEFOLD	(1 << 15) /* Casefolded file */
 #define S_VERITY	(1 << 16) /* Verity file (using fs/verity/) */
 #define S_KERNEL_FILE	(1 << 17) /* File is in use by the kernel (eg. fs/cachefiles) */
+#define S_NO_CASEFOLD	(1 << 18) /* Directory, and all descendents, are not casefolded */
 
 /*
  * Note that nosuid etc flags are inode-specific: setting some file-system
-- 
2.49.0

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 20, 2025 at 04:33:14PM +0200, Amir Goldstein wrote:
> I am saying that IMO a smaller impact (and less user friendly) fix is more
> appropriate way to deal with this problem.

What do you think about doing your approach as a stopgap?

It seems this is hitting a lot of people, something we can backport to
6.15 would be good to have.

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

On Fri, May 23, 2025 at 4:10 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Tue, May 20, 2025 at 04:33:14PM +0200, Amir Goldstein wrote:
> > I am saying that IMO a smaller impact (and less user friendly) fix is more
> > appropriate way to deal with this problem.
>
> What do you think about doing your approach as a stopgap?
>
> It seems this is hitting a lot of people, something we can backport to
> 6.15 would be good to have.

Yes, I think I can do that.
Will try to get to it next week.

Thanks,
Amir.

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

On Fri, May 23, 2025 at 7:14 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Fri, May 23, 2025 at 4:10 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Tue, May 20, 2025 at 04:33:14PM +0200, Amir Goldstein wrote:
> > > I am saying that IMO a smaller impact (and less user friendly) fix is more
> > > appropriate way to deal with this problem.
> >
> > What do you think about doing your approach as a stopgap?
> >
> > It seems this is hitting a lot of people, something we can backport to
> > 6.15 would be good to have.
>
> Yes, I think I can do that.
> Will try to get to it next week.
>

On second look, here is a compile-basic-sanity-tested patch.

Care to test if it does the job for you?

Thanks,
Amir.

[-- Attachment #2: 0001-ovl-support-casefolded-filesystems.patch --]
[-- Type: text/x-patch, Size: 3869 bytes --]

From 9fd9ce0e2b6de84b3665ed743b36348006739cd3 Mon Sep 17 00:00:00 2001
From: Amir Goldstein <amir73il@gmail.com>
Date: Fri, 23 May 2025 22:13:18 +0200
Subject: [PATCH] ovl: support casefolded filesystems

Case folding is often applied to subtrees and not on an entire
filesystem.

Disallowing layers from filesystems that support case folding is over
limiting.

Replace the rule that case-folding capable are not allowed as layers
with a rule that case folded directories are not allowed in a merged
directory stack.

Should case folding be enabled on an underlying directory while
overlayfs is mounted the outcome is generally undefined.

Specifically in ovl_lookup(), we check the base underlying directory
and fail with -ESTALE if an underlying directory case folding is
enabled.

Suggested-by: Kent Overstreet <kent.overstreet@linux.dev>
Link: https://lore.kernel.org/linux-fsdevel/20250520051600.1903319-1-kent.overstreet@linux.dev/
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c  | 10 ++++++++++
 fs/overlayfs/params.c | 10 ++++------
 fs/overlayfs/util.c   | 15 +++++++++++----
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index d489e80feb6f..5168b72d9710 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -237,6 +237,16 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
 	bool is_upper = d->layer->idx == 0;
 	char val;
 
+	/*
+	 * We allow filesystems that are case-folding capable but deny composing
+	 * ovl stack from case-folded directories. If someone has enabled case
+	 * folding on a directory on underlying layer, the warranty of the ovl
+	 * stack is voided.
+	 */
+	err = -ESTALE;
+	if (sb_has_encoding(base->d_sb) && IS_CASEFOLDED(d_inode(base)))
+		goto out;
+
 	this = ovl_lookup_positive_unlocked(d, name, base, namelen, drop_negative);
 	if (IS_ERR(this)) {
 		err = PTR_ERR(this);
diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
index f42488c01957..30cd7145b234 100644
--- a/fs/overlayfs/params.c
+++ b/fs/overlayfs/params.c
@@ -282,13 +282,11 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
 		return invalfc(fc, "%s is not a directory", name);
 
 	/*
-	 * Root dentries of case-insensitive capable filesystems might
-	 * not have the dentry operations set, but still be incompatible
-	 * with overlayfs.  Check explicitly to prevent post-mount
-	 * failures.
+	 * Allow filesystems that are case-folding capable but deny composing
+	 * ovl stack from case-folded directories.
 	 */
-	if (sb_has_encoding(path->mnt->mnt_sb))
-		return invalfc(fc, "case-insensitive capable filesystem on %s not supported", name);
+	if (sb_has_encoding(path->mnt->mnt_sb) && IS_CASEFOLDED(d_inode(path->dentry)))
+		return invalfc(fc, "case-insensitive directory on %s not supported", name);
 
 	if (ovl_dentry_weird(path->dentry))
 		return invalfc(fc, "filesystem on %s not supported", name);
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index dcccb4b4a66c..593c4da107d6 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -206,10 +206,17 @@ bool ovl_dentry_weird(struct dentry *dentry)
 	if (!d_can_lookup(dentry) && !d_is_file(dentry) && !d_is_symlink(dentry))
 		return true;
 
-	return dentry->d_flags & (DCACHE_NEED_AUTOMOUNT |
-				  DCACHE_MANAGE_TRANSIT |
-				  DCACHE_OP_HASH |
-				  DCACHE_OP_COMPARE);
+	if (dentry->d_flags & (DCACHE_NEED_AUTOMOUNT | DCACHE_MANAGE_TRANSIT))
+		return true;
+
+	/*
+	 * Allow filesystems that are case-folding capable but deny composing
+	 * ovl stack from case-folded directories.
+	 */
+	if (sb_has_encoding(dentry->d_sb))
+		return IS_CASEFOLDED(d_inode(dentry));
+
+	return dentry->d_flags & (DCACHE_OP_HASH | DCACHE_OP_COMPARE);
 }
 
 enum ovl_path_type ovl_path_type(struct dentry *dentry)
-- 
2.34.1

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:

That makes fstests generic/631 pass.

We really should be logging something in dmesg on error due to
casefolded directory, though.

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

On Fri, May 23, 2025 at 11:10 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
>
> That makes fstests generic/631 pass.

Yes, that is not very surprising.
I meant if you could help test that:

1. mounting case folder upperdir/lowerdir fails
2. lookup a case folder subdir fails
3. lookup in a dir that was empty and became case folder while ovl was
mounted fails

For me, I do not have any setup with case folding subtrees
so testing those use cases would take me time and
I think that you must have tested all those scenarios with your patch set?
and maybe already have some fstests for them?

>
> We really should be logging something in dmesg on error due to
> casefolded directory, though.

No problem.
Attached v2 with warnings.

For example, here is the new warning in test overlay/065:

[  131.815110] overlayfs: failed lookup in lower (/lower,
name='upper', err=-40): overlapping layers

Thanks,
Amir.

[-- Attachment #2: v2-0001-ovl-support-layers-on-case-folding-capable-filesy.patch --]
[-- Type: text/x-patch, Size: 5181 bytes --]

From 4f730811b80670b65e4edd818621b988c59e150b Mon Sep 17 00:00:00 2001
From: Amir Goldstein <amir73il@gmail.com>
Date: Fri, 23 May 2025 22:13:18 +0200
Subject: [PATCH v2] ovl: support layers on case-folding capable filesystems

Case folding is often applied to subtrees and not on an entire
filesystem.

Disallowing layers from filesystems that support case folding is over
limiting.

Replace the rule that case-folding capable are not allowed as layers
with a rule that case folded directories are not allowed in a merged
directory stack.

Should case folding be enabled on an underlying directory while
overlayfs is mounted the outcome is generally undefined.

Specifically in ovl_lookup(), we check the base underlying directory
and fail with -ESTALE and write a warning to kmsg if an underlying
directory case folding is enabled.

Suggested-by: Kent Overstreet <kent.overstreet@linux.dev>
Link: https://lore.kernel.org/linux-fsdevel/20250520051600.1903319-1-kent.overstreet@linux.dev/
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c  | 25 ++++++++++++++++++++++---
 fs/overlayfs/params.c | 11 +++++------
 fs/overlayfs/util.c   | 15 +++++++++++----
 3 files changed, 38 insertions(+), 13 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index d489e80feb6f..333017f6ae65 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -230,13 +230,26 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
 			     struct dentry **ret, bool drop_negative)
 {
 	struct ovl_fs *ofs = OVL_FS(d->sb);
-	struct dentry *this;
+	struct dentry *this = NULL;
+	const char *warn;
 	struct path path;
 	int err;
 	bool last_element = !post[0];
 	bool is_upper = d->layer->idx == 0;
 	char val;
 
+	/*
+	 * We allow filesystems that are case-folding capable but deny composing
+	 * ovl stack from case-folded directories. If someone has enabled case
+	 * folding on a directory on underlying layer, the warranty of the ovl
+	 * stack is voided.
+	 */
+	if (sb_has_encoding(base->d_sb) && IS_CASEFOLDED(d_inode(base))) {
+		warn = "case folded parent";
+		err = -ESTALE;
+		goto out_warn;
+	}
+
 	this = ovl_lookup_positive_unlocked(d, name, base, namelen, drop_negative);
 	if (IS_ERR(this)) {
 		err = PTR_ERR(this);
@@ -248,8 +261,9 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
 
 	if (ovl_dentry_weird(this)) {
 		/* Don't support traversing automounts and other weirdness */
+		warn = "unsupported object type";
 		err = -EREMOTE;
-		goto out_err;
+		goto out_warn;
 	}
 
 	path.dentry = this;
@@ -283,8 +297,9 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
 	} else {
 		if (ovl_lookup_trap_inode(d->sb, this)) {
 			/* Caught in a trap of overlapping layers */
+			warn = "overlapping layers";
 			err = -ELOOP;
-			goto out_err;
+			goto out_warn;
 		}
 
 		if (last_element)
@@ -316,6 +331,10 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,
 	this = NULL;
 	goto out;
 
+out_warn:
+	pr_warn_ratelimited("failed lookup in %s (%pd2, name='%.*s', err=%i): %s\n",
+			    is_upper ? "upper" : "lower", base,
+			    namelen, name, err, warn);
 out_err:
 	dput(this);
 	return err;
diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
index f42488c01957..848a6b135143 100644
--- a/fs/overlayfs/params.c
+++ b/fs/overlayfs/params.c
@@ -282,13 +282,12 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
 		return invalfc(fc, "%s is not a directory", name);
 
 	/*
-	 * Root dentries of case-insensitive capable filesystems might
-	 * not have the dentry operations set, but still be incompatible
-	 * with overlayfs.  Check explicitly to prevent post-mount
-	 * failures.
+	 * Allow filesystems that are case-folding capable but deny composing
+	 * ovl stack from case-folded directories.
 	 */
-	if (sb_has_encoding(path->mnt->mnt_sb))
-		return invalfc(fc, "case-insensitive capable filesystem on %s not supported", name);
+	if (sb_has_encoding(path->mnt->mnt_sb) &&
+	    IS_CASEFOLDED(d_inode(path->dentry)))
+		return invalfc(fc, "case-insensitive directory on %s not supported", name);
 
 	if (ovl_dentry_weird(path->dentry))
 		return invalfc(fc, "filesystem on %s not supported", name);
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index dcccb4b4a66c..593c4da107d6 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -206,10 +206,17 @@ bool ovl_dentry_weird(struct dentry *dentry)
 	if (!d_can_lookup(dentry) && !d_is_file(dentry) && !d_is_symlink(dentry))
 		return true;
 
-	return dentry->d_flags & (DCACHE_NEED_AUTOMOUNT |
-				  DCACHE_MANAGE_TRANSIT |
-				  DCACHE_OP_HASH |
-				  DCACHE_OP_COMPARE);
+	if (dentry->d_flags & (DCACHE_NEED_AUTOMOUNT | DCACHE_MANAGE_TRANSIT))
+		return true;
+
+	/*
+	 * Allow filesystems that are case-folding capable but deny composing
+	 * ovl stack from case-folded directories.
+	 */
+	if (sb_has_encoding(dentry->d_sb))
+		return IS_CASEFOLDED(d_inode(dentry));
+
+	return dentry->d_flags & (DCACHE_OP_HASH | DCACHE_OP_COMPARE);
 }
 
 enum ovl_path_type ovl_path_type(struct dentry *dentry)
-- 
2.34.1

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Sat, May 24, 2025 at 03:01:44PM +0200, Amir Goldstein wrote:
> On Fri, May 23, 2025 at 11:10 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
> >
> > That makes fstests generic/631 pass.
> 
> Yes, that is not very surprising.
> I meant if you could help test that:
> 
> 1. mounting case folder upperdir/lowerdir fails
> 2. lookup a case folder subdir fails
> 3. lookup in a dir that was empty and became case folder while ovl was
> mounted fails
> 
> For me, I do not have any setup with case folding subtrees
> so testing those use cases would take me time and
> I think that you must have tested all those scenarios with your patch set?
> and maybe already have some fstests for them?

Unmount fauls after I test an overlayfs with a casefold subdir:

Testing an overlayfs on a casefold fs with non-casefolded dirs
Test using casefolded dir - should fail
overlayfs: failed to resolve '/mnt/casefold': -2
mount: /mnt/merged: special device overlay does not exist.
       dmesg(1) may have more information after failed mount system call.
Test using a dir with a casefold subdir - should mount
overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
overlayfs: workdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
ls: cannot access '/mnt/merged/dir/casefold': No such file or directory
umount: /mnt: target is busy.

https://evilpiepirate.org/git/ktest.git/commit/?id=47d1f2a04d79bc4cbc843f81e71eb7d821fb8384

Re: [PATCH 0/6] overlayfs + casefolding

[Amir Goldstein]

On Sun, May 25, 2025 at 8:27 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Sat, May 24, 2025 at 03:01:44PM +0200, Amir Goldstein wrote:
> > On Fri, May 23, 2025 at 11:10 PM Kent Overstreet
> > <kent.overstreet@linux.dev> wrote:
> > >
> > > On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
> > >
> > > That makes fstests generic/631 pass.
> >
> > Yes, that is not very surprising.
> > I meant if you could help test that:
> >
> > 1. mounting case folder upperdir/lowerdir fails
> > 2. lookup a case folder subdir fails
> > 3. lookup in a dir that was empty and became case folder while ovl was
> > mounted fails
> >
> > For me, I do not have any setup with case folding subtrees
> > so testing those use cases would take me time and
> > I think that you must have tested all those scenarios with your patch set?
> > and maybe already have some fstests for them?
>
> Unmount fauls after I test an overlayfs with a casefold subdir:
>
> Testing an overlayfs on a casefold fs with non-casefolded dirs
> Test using casefolded dir - should fail
> overlayfs: failed to resolve '/mnt/casefold': -2
> mount: /mnt/merged: special device overlay does not exist.
>        dmesg(1) may have more information after failed mount system call.

Test is using the wrong path:


+    echo "Test using casefolded dir - should fail"
+    ! mount -t overlay -o
lowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay
/mnt/merged
+    ! mount -t overlay -o
lowerdir=/mnt/casefold,upperdir=/mnt/casefold,workdir=/mnt/work
overlay /mnt/merged

There is no "/mnt/casefold"

> Test using a dir with a casefold subdir - should mount
> overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
> overlayfs: workdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.

Those warnings are because you have a stray mount command above:
+    echo "Test using casefolded dir - should fail"
+    ! mount -t overlay -o
lowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay
/mnt/merged

So a mount already exists. leftover?

> ls: cannot access '/mnt/merged/dir/casefold': No such file or directory
> umount: /mnt: target is busy.

Not sure about that, but could be due to the aforementioned stay mount.

>
> https://evilpiepirate.org/git/ktest.git/commit/?id=47d1f2a04d79bc4cbc843f81e71eb7d821fb8384

Please fix the test and report Tested-by if all works as expected.
Please include dmesg so we can see the new warnings.

Thanks,
Amir.

Re: [PATCH 0/6] overlayfs + casefolding

[Kent Overstreet]

On Tue, May 27, 2025 at 10:57:05AM +0200, Amir Goldstein wrote:
> On Sun, May 25, 2025 at 8:27 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Sat, May 24, 2025 at 03:01:44PM +0200, Amir Goldstein wrote:
> > > On Fri, May 23, 2025 at 11:10 PM Kent Overstreet
> > > <kent.overstreet@linux.dev> wrote:
> > > >
> > > > On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
> > > >
> > > > That makes fstests generic/631 pass.
> > >
> > > Yes, that is not very surprising.
> > > I meant if you could help test that:
> > >
> > > 1. mounting case folder upperdir/lowerdir fails
> > > 2. lookup a case folder subdir fails
> > > 3. lookup in a dir that was empty and became case folder while ovl was
> > > mounted fails
> > >
> > > For me, I do not have any setup with case folding subtrees
> > > so testing those use cases would take me time and
> > > I think that you must have tested all those scenarios with your patch set?
> > > and maybe already have some fstests for them?
> >
> > Unmount fauls after I test an overlayfs with a casefold subdir:
> >
> > Testing an overlayfs on a casefold fs with non-casefolded dirs
> > Test using casefolded dir - should fail
> > overlayfs: failed to resolve '/mnt/casefold': -2
> > mount: /mnt/merged: special device overlay does not exist.
> >        dmesg(1) may have more information after failed mount system call.
> 
> Test is using the wrong path:
> 
> 
> +    echo "Test using casefolded dir - should fail"
> +    ! mount -t overlay -o
> lowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay
> /mnt/merged
> +    ! mount -t overlay -o
> lowerdir=/mnt/casefold,upperdir=/mnt/casefold,workdir=/mnt/work
> overlay /mnt/merged
> 
> There is no "/mnt/casefold"

*nod*

> > Test using a dir with a casefold subdir - should mount
> > overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
> > overlayfs: workdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
> 
> Those warnings are because you have a stray mount command above:
> +    echo "Test using casefolded dir - should fail"
> +    ! mount -t overlay -o
> lowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay
> /mnt/merged
> 
> So a mount already exists. leftover?

*snort* - -ENOCAFFEINE, I presume.

The new warning doesn't fire after the last mount.

ls: cannot access '/mnt/merged/casefold': Object is remote

But nothing in dmesg. Adding a printk to ovl_mount_dir_check() shows
that it's never called for the 'ls /mnt/merged/casefold' call.