[RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[André Almeida]

Hi everyone,

When using overlayfs with the mount option index=on, the first time a directory is
used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
filesystem being used in the layers. If the upper dir is reused, overlayfs
refuses to mount for a different filesystem, by comparing the UUID with what's
stored at overlay.origin, and it fails with "failed to verify upper root origin"
on dmesg. Remounting with the very same fs is supported and works fine.

However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
disk image with btrfs, a random UUID is assigned for the following disks each
time they are mounted, stored at temp_fsid and used across the kernel as the
disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
the original (and duplicated) UUID for all disks.

This feature doesn't work well with overlayfs with index=on, as when the image
is mounted a second time, will get a different UUID and ovl will refuse to
mount, breaking the user expectation that using the same image should work. A
small script can be find in the end of this cover letter that illustrates this.

From this, I can think of some options:

- Use statfs() internally to always get the fsid, that is persistent. The patch
here illustrates that approach, but doesn't fully implement it.
- Create a new sb op, called get_uuid() so the filesystem returns what's
appropriated.
- Have a workaround in ovl for btrfs.
- Document this as unsupported, and userland needs to erase overlay.origin each
time it wants to remount.
- If ovl detects that temp_fsid and index are being used at the same time,
refuses to mount.

I'm not sure which one would be better here, so I would like to hear some ideas
on this.

Thanks!
	André

---

To reproduce:

mkdir -p dir1 dir2

fallocate -l 300m ./disk1.img
mkfs.btrfs -q -f ./disk1.img

# cloning the disks
cp disk1.img disk2.img
sudo mount -o loop ./disk1.img dir1
sudo mount -o loop ./disk2.img dir2

mkdir -p dir2/lower aux/upper aux/work

# this works
sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower

sudo umount dir2/lower
sudo umount dir2

sudo mount -o loop ./disk2.img dir2

# this doesn't works
sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower

André Almeida (1):
  ovl: Use fsid as unique identifier for trusted origin

 fs/overlayfs/copy_up.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

-- 
2.51.0

[RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[André Almeida]

Some filesystem have non-persistent UUIDs, that can change between
mounting, even if the filesystem is not modified. To prevent
false-positives when mounting overlayfs with index enabled, use the fsid
reported from statfs that is persistent across mounts.

Signed-off-by: André Almeida <andrealmeid@igalia.com>
---
This patch is just for illustrative purposes and doesn't work.
---
 fs/overlayfs/copy_up.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
index aac7e34f56c1..633d9470a089 100644
--- a/fs/overlayfs/copy_up.c
+++ b/fs/overlayfs/copy_up.c
@@ -8,6 +8,7 @@
 #include <linux/fs.h>
 #include <linux/slab.h>
 #include <linux/file.h>
+#include <linux/statfs.h>
 #include <linux/fileattr.h>
 #include <linux/splice.h>
 #include <linux/xattr.h>
@@ -421,9 +422,14 @@ struct ovl_fh *ovl_encode_real_fh(struct ovl_fs *ofs, struct inode *realinode,
 	struct ovl_fh *fh;
 	int fh_type, dwords;
 	int buflen = MAX_HANDLE_SZ;
-	uuid_t *uuid = &realinode->i_sb->s_uuid;
+	uuid_t uuid;
+	struct kstatfs ks;
 	int err;
 
+	// RFC: dentry can't be NULL, uuid needs a type cast
+	realinode->i_sb->s_op->statfs(NULL, &ks);
+	uuid.b = ks.f_fsid;
+
 	/* Make sure the real fid stays 32bit aligned */
 	BUILD_BUG_ON(OVL_FH_FID_OFFSET % 4);
 	BUILD_BUG_ON(MAX_HANDLE_SZ + OVL_FH_FID_OFFSET > 255);
-- 
2.51.0

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[Christoph Hellwig]

On Mon, Oct 13, 2025 at 10:57:07PM -0300, André Almeida wrote:
> Some filesystem have non-persistent UUIDs, that can change between
> mounting, even if the filesystem is not modified. To prevent
> false-positives when mounting overlayfs with index enabled, use the fsid
> reported from statfs that is persistent across mounts.

Please fix btrfs to not change uuids, as that completely defeats the
point of uuids.

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[Qu Wenruo]

在 2025/10/14 15:09, Christoph Hellwig 写道:
> On Mon, Oct 13, 2025 at 10:57:07PM -0300, André Almeida wrote:
>> Some filesystem have non-persistent UUIDs, that can change between
>> mounting, even if the filesystem is not modified. To prevent
>> false-positives when mounting overlayfs with index enabled, use the fsid
>> reported from statfs that is persistent across mounts.
> 
> Please fix btrfs to not change uuids, as that completely defeats the
> point of uuids.
> 

That is the temp-fsid feature from Anand, introduced by commit 
a5b8a5f9f835 ("btrfs: support cloned-device mount capability").

I'm not 100% sure if it's really that important to support mounting 
cloned devices in the first place, as LVM will reject activating any LVs 
if there is even conflicting VGs names, not to mention conflicting UUIDs.

If temp-fsid is causing problems with overlayfs, I'm happy to remove it, 
as this really looks like a niche that no one is asking.

Yes, mounting cloned devices can be useful for certain cases, but with 
metadata_uuid changing the uuid should not even take a second, or one 
can just unregister the previously scanned device.

I'd say we paid too much cost for a niche that is not worthy.

Thanks,
Qu

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[David Sterba]

On Tue, Oct 14, 2025 at 03:43:54PM +1030, Qu Wenruo wrote:
> 在 2025/10/14 15:09, Christoph Hellwig 写道:
> > On Mon, Oct 13, 2025 at 10:57:07PM -0300, André Almeida wrote:
> >> Some filesystem have non-persistent UUIDs, that can change between
> >> mounting, even if the filesystem is not modified. To prevent
> >> false-positives when mounting overlayfs with index enabled, use the fsid
> >> reported from statfs that is persistent across mounts.
> > 
> > Please fix btrfs to not change uuids, as that completely defeats the
> > point of uuids.
> > 
> 
> That is the temp-fsid feature from Anand, introduced by commit 
> a5b8a5f9f835 ("btrfs: support cloned-device mount capability").
> 
> I'm not 100% sure if it's really that important to support mounting 
> cloned devices in the first place, as LVM will reject activating any LVs 
> if there is even conflicting VGs names, not to mention conflicting UUIDs.
> 
> If temp-fsid is causing problems with overlayfs, I'm happy to remove it, 
> as this really looks like a niche that no one is asking.

What do you mean no one asking?  This was specifically asked for by
Steam to do A/B root partition mounts for recovery. It is a niche use
case but it has its users.

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[André Almeida]

On 10/14/25 14:40, David Sterba wrote:
> On Tue, Oct 14, 2025 at 03:43:54PM +1030, Qu Wenruo wrote:
>> 在 2025/10/14 15:09, Christoph Hellwig 写道:
>>> On Mon, Oct 13, 2025 at 10:57:07PM -0300, André Almeida wrote:
>>>> Some filesystem have non-persistent UUIDs, that can change between
>>>> mounting, even if the filesystem is not modified. To prevent
>>>> false-positives when mounting overlayfs with index enabled, use the fsid
>>>> reported from statfs that is persistent across mounts.
>>> Please fix btrfs to not change uuids, as that completely defeats the
>>> point of uuids.
>>>
>> That is the temp-fsid feature from Anand, introduced by commit
>> a5b8a5f9f835 ("btrfs: support cloned-device mount capability").
>>
>> I'm not 100% sure if it's really that important to support mounting
>> cloned devices in the first place, as LVM will reject activating any LVs
>> if there is even conflicting VGs names, not to mention conflicting UUIDs.
>>
>> If temp-fsid is causing problems with overlayfs, I'm happy to remove it,
>> as this really looks like a niche that no one is asking.
> What do you mean no one asking?  This was specifically asked for by
> Steam to do A/B root partition mounts for recovery. It is a niche use
> case but it has its users.
That's right, I've come across the issue reported here while working 
with SteamOS partitions, so it's being used. The original thread for 
this feature has more information about the use case: 
https://lore.kernel.org/linux-btrfs/20230504170708.787361-1-gpiccoli@igalia.com/ 

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[Anand Jain]

On 14-Oct-25 12:39 PM, Christoph Hellwig wrote:
> On Mon, Oct 13, 2025 at 10:57:07PM -0300, André Almeida wrote:
>> Some filesystem have non-persistent UUIDs, that can change between
>> mounting, even if the filesystem is not modified. To prevent
>> false-positives when mounting overlayfs with index enabled, use the fsid
>> reported from statfs that is persistent across mounts.
> 
> Please fix btrfs to not change uuids, as that completely defeats the
> point of uuids.

We needed cloned device mount support for an A/B testing
use case, but changing the on-disk UUID defeats the purpose.

Right now, ext4 and Btrfs can mount identical devices,
but XFS can't. How about extending this to the common
VFS layer and adding a parameter to tell apart a cloned
device from the same device accessed through multiple
paths? I haven't looked into the details yet, but I can
dig it further.

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[Christoph Hellwig]

On Wed, Oct 15, 2025 at 07:46:34AM +0800, Anand Jain wrote:
> We needed cloned device mount support for an A/B testing
> use case, but changing the on-disk UUID defeats the purpose.
> 
> Right now, ext4 and Btrfs can mount identical devices,
> but XFS can't. How about extending this to the common
> VFS layer and adding a parameter to tell apart a cloned
> device from the same device accessed through multiple
> paths? I haven't looked into the details yet, but I can
> dig it further.

If you clone a device you need to change the user visible uuid/fsid,
and you need to do that explicitly to a known either saved or user
controlled value.  Assigning a random ID is highly dangerous as seen
here.

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[Dave Chinner]

On Wed, Oct 15, 2025 at 07:46:34AM +0800, Anand Jain wrote:
> On 14-Oct-25 12:39 PM, Christoph Hellwig wrote:
> > On Mon, Oct 13, 2025 at 10:57:07PM -0300, André Almeida wrote:
> > > Some filesystem have non-persistent UUIDs, that can change
> > > between mounting, even if the filesystem is not modified. To
> > > prevent false-positives when mounting overlayfs with index
> > > enabled, use the fsid reported from statfs that is persistent
> > > across mounts.
> > 
> > Please fix btrfs to not change uuids, as that completely defeats
> > the point of uuids.
> 
> We needed cloned device mount support for an A/B testing use case,
> but changing the on-disk UUID defeats the purpose.
> 
> Right now, ext4 and Btrfs can mount identical devices, but XFS
> can't.

Absolutely not true.

XFS has been able to mount filesystems with duplicate UUIDs on Linux
for almost 25 years. The "-o nouuid" mount option (introduced in
2001) to bypass the duplicate uuid checks done at mount time.

XFS tracks all mounted filesystem UUIDs largely to prevent multiple
mounts of the same filesystem due to multipath storage presenting it
via multiple different block devices.

The nouuid mount option was added back when enterprise storage
arrays started supporting hardware level thinp and LUN
clone/snapshot functionality. Adding "-o nouuid" allowed cloned LUNs
to be mounted for for backup/recovery purposes whilst the main
filesystem was still mounted and in active use.

> How about extending this to the common
> VFS layer and adding a parameter to tell apart a cloned
> device from the same device accessed through multiple
> paths?

Perhaps we should lift the XFS UUID tracking code to the VFS
and intercept "-o nouuid" at the VFS to allow duplicates only when
that mount option is set?

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[Anand Jain]

On 21/10/25 05:43, Dave Chinner wrote:
> On Wed, Oct 15, 2025 at 07:46:34AM +0800, Anand Jain wrote:
>> On 14-Oct-25 12:39 PM, Christoph Hellwig wrote:
>>> On Mon, Oct 13, 2025 at 10:57:07PM -0300, André Almeida wrote:
>>>> Some filesystem have non-persistent UUIDs, that can change
>>>> between mounting, even if the filesystem is not modified. To
>>>> prevent false-positives when mounting overlayfs with index
>>>> enabled, use the fsid reported from statfs that is persistent
>>>> across mounts.
>>>
>>> Please fix btrfs to not change uuids, as that completely defeats
>>> the point of uuids.
>>
>> We needed cloned device mount support for an A/B testing use case,
>> but changing the on-disk UUID defeats the purpose.
>>
>> Right now, ext4 and Btrfs can mount identical devices, but XFS
>> can't.
> 
> Absolutely not true.
> 
> XFS has been able to mount filesystems with duplicate UUIDs on Linux
> for almost 25 years. The "-o nouuid" mount option (introduced in
> 2001) to bypass the duplicate uuid checks done at mount time.
> 

Damn, I completely missed the nouuid XFS option. My bad!!
> XFS tracks all mounted filesystem UUIDs largely to prevent multiple
> mounts of the same filesystem due to multipath storage presenting it
> via multiple different block devices.
 > > The nouuid mount option was added back when enterprise storage
> arrays started supporting hardware level thinp and LUN
> clone/snapshot functionality. Adding "-o nouuid" allowed cloned LUNs
> to be mounted for for backup/recovery purposes whilst the main
> filesystem was still mounted and in active use.
> 

Agree. Also, in some SAN error situations, the same device may
disappear and reappear with a new maj:min.

>> How about extending this to the common
>> VFS layer and adding a parameter to tell apart a cloned
>> device from the same device accessed through multiple
>> paths?
> 
> Perhaps we should lift the XFS UUID tracking code to the VFS
> and intercept "-o nouuid" at the VFS to allow duplicates only when
> that mount option is set?
> 
> -Dave.

It looks like XFS (with -o nouuid) and ext4 allow duplicate
FSIDs to pass into VFS. We may be able to extend this to Btrfs,
though there could be conflicts with fanotify? I'm not sure yet.
we still need -o nouuid, to alert admins to handle cases where a
device reappears with a new devt. I'm digging more.

Thanks, Anand

Re: [RFC PATCH 1/1] ovl: Use fsid as unique identifier for trusted origin

[Amir Goldstein]

On Tue, Oct 14, 2025 at 3:57 AM André Almeida <andrealmeid@igalia.com> wrote:
>
> Some filesystem have non-persistent UUIDs, that can change between
> mounting, even if the filesystem is not modified. To prevent
> false-positives when mounting overlayfs with index enabled, use the fsid
> reported from statfs that is persistent across mounts.
>
> Signed-off-by: André Almeida <andrealmeid@igalia.com>
> ---
> This patch is just for illustrative purposes and doesn't work.
> ---
>  fs/overlayfs/copy_up.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
> index aac7e34f56c1..633d9470a089 100644
> --- a/fs/overlayfs/copy_up.c
> +++ b/fs/overlayfs/copy_up.c
> @@ -8,6 +8,7 @@
>  #include <linux/fs.h>
>  #include <linux/slab.h>
>  #include <linux/file.h>
> +#include <linux/statfs.h>
>  #include <linux/fileattr.h>
>  #include <linux/splice.h>
>  #include <linux/xattr.h>
> @@ -421,9 +422,14 @@ struct ovl_fh *ovl_encode_real_fh(struct ovl_fs *ofs, struct inode *realinode,
>         struct ovl_fh *fh;
>         int fh_type, dwords;
>         int buflen = MAX_HANDLE_SZ;
> -       uuid_t *uuid = &realinode->i_sb->s_uuid;
> +       uuid_t uuid;
> +       struct kstatfs ks;
>         int err;
>
> +       // RFC: dentry can't be NULL, uuid needs a type cast
> +       realinode->i_sb->s_op->statfs(NULL, &ks);
> +       uuid.b = ks.f_fsid;
> +

Just wanted to add that from overlayfs POV, this is completely wrong,
because there are many fs whose fsid is not persistent including the
default fs that are used in many distros and the whole idea of the
origin xattr value is that it needs to be a persistent descriptor of the
lower layer directory.

Thanks,
Amir.

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[Qu Wenruo]

在 2025/10/14 12:27, André Almeida 写道:
> Hi everyone,
> 
> When using overlayfs with the mount option index=on, the first time a directory is
> used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
> filesystem being used in the layers. If the upper dir is reused, overlayfs
> refuses to mount for a different filesystem, by comparing the UUID with what's
> stored at overlay.origin, and it fails with "failed to verify upper root origin"
> on dmesg. Remounting with the very same fs is supported and works fine.
> 
> However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
> disk image with btrfs, a random UUID is assigned for the following disks each
> time they are mounted, stored at temp_fsid and used across the kernel as the
> disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
> the original (and duplicated) UUID for all disks.

Yep, that's the btrfs' hack to allowing mounting cloned devices (as long 
as they are all single-device only btrfs)

Although I'm not a huge fan for that, without that you can not even 
mount any cloned btrfs in the first place.

> 
> This feature doesn't work well with overlayfs with index=on, as when the image
> is mounted a second time, will get a different UUID and ovl will refuse to
> mount, breaking the user expectation that using the same image should work. A
> small script can be find in the end of this cover letter that illustrates this.
> 
>  From this, I can think of some options:
> 
> - Use statfs() internally to always get the fsid, that is persistent. The patch
> here illustrates that approach, but doesn't fully implement it.
> - Create a new sb op, called get_uuid() so the filesystem returns what's
> appropriated.
> - Have a workaround in ovl for btrfs.
> - Document this as unsupported, and userland needs to erase overlay.origin each
> time it wants to remount.
> - If ovl detects that temp_fsid and index are being used at the same time,
> refuses to mount.

Or, let btrfs to reject the cloned device in the first place.

> 
> I'm not sure which one would be better here, so I would like to hear some ideas
> on this.
> 
> Thanks!
> 	André
> 
> ---
> 
> To reproduce:
> 
> mkdir -p dir1 dir2
> 
> fallocate -l 300m ./disk1.img
> mkfs.btrfs -q -f ./disk1.img
> 
> # cloning the disks
> cp disk1.img disk2.img

If you really want to use the same copied fs, at least you can use
`btrfstune -m disk2.img` to change it to a new metadata uuid (without 
re-writing all metadata).

Then everything should work.

Thanks,
Qu
> sudo mount -o loop ./disk1.img dir1
> sudo mount -o loop ./disk2.img dir2
> 
> mkdir -p dir2/lower aux/upper aux/work
> 
> # this works
> sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower
> 
> sudo umount dir2/lower
> sudo umount dir2
> 
> sudo mount -o loop ./disk2.img dir2
> 
> # this doesn't works
> sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower
> 
> André Almeida (1):
>    ovl: Use fsid as unique identifier for trusted origin
> 
>   fs/overlayfs/copy_up.c | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
> 

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[David Sterba]

On Mon, Oct 13, 2025 at 10:57:06PM -0300, André Almeida wrote:
> Hi everyone,
> 
> When using overlayfs with the mount option index=on, the first time a directory is
> used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
> filesystem being used in the layers. If the upper dir is reused, overlayfs
> refuses to mount for a different filesystem, by comparing the UUID with what's
> stored at overlay.origin, and it fails with "failed to verify upper root origin"
> on dmesg. Remounting with the very same fs is supported and works fine.
> 
> However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
> disk image with btrfs, a random UUID is assigned for the following disks each
> time they are mounted, stored at temp_fsid and used across the kernel as the
> disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
> the original (and duplicated) UUID for all disks.
> 
> This feature doesn't work well with overlayfs with index=on, as when the image
> is mounted a second time, will get a different UUID and ovl will refuse to
> mount, breaking the user expectation that using the same image should work. A
> small script can be find in the end of this cover letter that illustrates this.
> 
> >From this, I can think of some options:
> 
> - Use statfs() internally to always get the fsid, that is persistent. The patch
> here illustrates that approach, but doesn't fully implement it.
> - Create a new sb op, called get_uuid() so the filesystem returns what's
> appropriated.
> - Have a workaround in ovl for btrfs.
> - Document this as unsupported, and userland needs to erase overlay.origin each
> time it wants to remount.
> - If ovl detects that temp_fsid and index are being used at the same time,
> refuses to mount.
> 
> I'm not sure which one would be better here, so I would like to hear some ideas
> on this.

I haven't looked deeper if there's a workable solution, but the feature
combination should be refused. I don't think this will affect many
users.

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[Qu Wenruo]

在 2025/10/15 04:54, David Sterba 写道:
> On Mon, Oct 13, 2025 at 10:57:06PM -0300, André Almeida wrote:
>> Hi everyone,
>>
>> When using overlayfs with the mount option index=on, the first time a directory is
>> used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
>> filesystem being used in the layers. If the upper dir is reused, overlayfs
>> refuses to mount for a different filesystem, by comparing the UUID with what's
>> stored at overlay.origin, and it fails with "failed to verify upper root origin"
>> on dmesg. Remounting with the very same fs is supported and works fine.
>>
>> However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
>> disk image with btrfs, a random UUID is assigned for the following disks each
>> time they are mounted, stored at temp_fsid and used across the kernel as the
>> disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
>> the original (and duplicated) UUID for all disks.
>>
>> This feature doesn't work well with overlayfs with index=on, as when the image
>> is mounted a second time, will get a different UUID and ovl will refuse to
>> mount, breaking the user expectation that using the same image should work. A
>> small script can be find in the end of this cover letter that illustrates this.
>>
>> >From this, I can think of some options:
>>
>> - Use statfs() internally to always get the fsid, that is persistent. The patch
>> here illustrates that approach, but doesn't fully implement it.
>> - Create a new sb op, called get_uuid() so the filesystem returns what's
>> appropriated.
>> - Have a workaround in ovl for btrfs.
>> - Document this as unsupported, and userland needs to erase overlay.origin each
>> time it wants to remount.
>> - If ovl detects that temp_fsid and index are being used at the same time,
>> refuses to mount.
>>
>> I'm not sure which one would be better here, so I would like to hear some ideas
>> on this.
> 
> I haven't looked deeper if there's a workable solution, but the feature
> combination should be refused. I don't think this will affect many
> users.
> 

I believe the root problem is that we're not fully implementing the 
proper handling just like other single-device fses.

We do not use on-disk flags which means at least one fsid is registered 
into btrfs, thus we have to use different temp-fsid.

If fully single-device feature flag is properly implemented, we should 
be able to return the same uuid without extra hacks thus solve the problem.

Thanks,
Qu

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[Anand Jain]

On 15-Oct-25 5:08 AM, Qu Wenruo wrote:
> 
> 
> 在 2025/10/15 04:54, David Sterba 写道:
>> On Mon, Oct 13, 2025 at 10:57:06PM -0300, André Almeida wrote:
>>> Hi everyone,
>>>
>>> When using overlayfs with the mount option index=on, the first time a 
>>> directory is
>>> used as upper dir, overlayfs stores in a xattr "overlay.origin" the 
>>> UUID of the
>>> filesystem being used in the layers. If the upper dir is reused, 
>>> overlayfs
>>> refuses to mount for a different filesystem, by comparing the UUID 
>>> with what's
>>> stored at overlay.origin, and it fails with "failed to verify upper 
>>> root origin"
>>> on dmesg. Remounting with the very same fs is supported and works fine.
>>>
>>> However, btrfs mounts may have volatiles UUIDs. When mounting the 
>>> exact same
>>> disk image with btrfs, a random UUID is assigned for the following 
>>> disks each
>>> time they are mounted, stored at temp_fsid and used across the kernel 
>>> as the
>>> disk UUID. `btrfs filesystem show` presents that. Calling statfs() 
>>> however shows
>>> the original (and duplicated) UUID for all disks.
>>>
>>> This feature doesn't work well with overlayfs with index=on, as when 
>>> the image
>>> is mounted a second time, will get a different UUID and ovl will 
>>> refuse to
>>> mount, breaking the user expectation that using the same image should 
>>> work. A
>>> small script can be find in the end of this cover letter that 
>>> illustrates this.
>>>
>>> >From this, I can think of some options:
>>>
>>> - Use statfs() internally to always get the fsid, that is persistent. 
>>> The patch
>>> here illustrates that approach, but doesn't fully implement it.
>>> - Create a new sb op, called get_uuid() so the filesystem returns what's
>>> appropriated.
>>> - Have a workaround in ovl for btrfs.
>>> - Document this as unsupported, and userland needs to erase 
>>> overlay.origin each
>>> time it wants to remount.
>>> - If ovl detects that temp_fsid and index are being used at the same 
>>> time,
>>> refuses to mount.
>>>
>>> I'm not sure which one would be better here, so I would like to hear 
>>> some ideas
>>> on this.
>>
>> I haven't looked deeper if there's a workable solution, but the feature
>> combination should be refused. I don't think this will affect many
>> users.
>>
> 
> I believe the root problem is that we're not fully implementing the 
> proper handling just like other single-device fses.
> 
> We do not use on-disk flags which means at least one fsid is registered 
> into btrfs, thus we have to use different temp-fsid.
> 
> If fully single-device feature flag is properly implemented, we should 
> be able to return the same uuid without extra hacks thus solve the problem.

I had looked into this some time ago. Some libs, like libblkid,
don't handle multi-device filesystems or cloned devices with
temp FSIDs very well. I'm aware of it.

I've been making some progress on fixing those cases, but it's
a bit extensive since we first need enough test coverage,
and recent reappear-device inline with that.

Let's see how we can support use cases with identical devices
(where changing the UUID isn't an option) and keep things
compatible with systemd and library tools.

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[Qu Wenruo]

在 2025/10/15 10:35, Anand Jain 写道:
> On 15-Oct-25 5:08 AM, Qu Wenruo wrote:
>>
>>
>> 在 2025/10/15 04:54, David Sterba 写道:
>>> On Mon, Oct 13, 2025 at 10:57:06PM -0300, André Almeida wrote:
>>>> Hi everyone,
>>>>
>>>> When using overlayfs with the mount option index=on, the first time 
>>>> a directory is
>>>> used as upper dir, overlayfs stores in a xattr "overlay.origin" the 
>>>> UUID of the
>>>> filesystem being used in the layers. If the upper dir is reused, 
>>>> overlayfs
>>>> refuses to mount for a different filesystem, by comparing the UUID 
>>>> with what's
>>>> stored at overlay.origin, and it fails with "failed to verify upper 
>>>> root origin"
>>>> on dmesg. Remounting with the very same fs is supported and works fine.
>>>>
>>>> However, btrfs mounts may have volatiles UUIDs. When mounting the 
>>>> exact same
>>>> disk image with btrfs, a random UUID is assigned for the following 
>>>> disks each
>>>> time they are mounted, stored at temp_fsid and used across the 
>>>> kernel as the
>>>> disk UUID. `btrfs filesystem show` presents that. Calling statfs() 
>>>> however shows
>>>> the original (and duplicated) UUID for all disks.
>>>>
>>>> This feature doesn't work well with overlayfs with index=on, as when 
>>>> the image
>>>> is mounted a second time, will get a different UUID and ovl will 
>>>> refuse to
>>>> mount, breaking the user expectation that using the same image 
>>>> should work. A
>>>> small script can be find in the end of this cover letter that 
>>>> illustrates this.
>>>>
>>>> >From this, I can think of some options:
>>>>
>>>> - Use statfs() internally to always get the fsid, that is 
>>>> persistent. The patch
>>>> here illustrates that approach, but doesn't fully implement it.
>>>> - Create a new sb op, called get_uuid() so the filesystem returns 
>>>> what's
>>>> appropriated.
>>>> - Have a workaround in ovl for btrfs.
>>>> - Document this as unsupported, and userland needs to erase 
>>>> overlay.origin each
>>>> time it wants to remount.
>>>> - If ovl detects that temp_fsid and index are being used at the same 
>>>> time,
>>>> refuses to mount.
>>>>
>>>> I'm not sure which one would be better here, so I would like to hear 
>>>> some ideas
>>>> on this.
>>>
>>> I haven't looked deeper if there's a workable solution, but the feature
>>> combination should be refused. I don't think this will affect many
>>> users.
>>>
>>
>> I believe the root problem is that we're not fully implementing the 
>> proper handling just like other single-device fses.
>>
>> We do not use on-disk flags which means at least one fsid is 
>> registered into btrfs, thus we have to use different temp-fsid.
>>
>> If fully single-device feature flag is properly implemented, we should 
>> be able to return the same uuid without extra hacks thus solve the 
>> problem.
> 
> I had looked into this some time ago. Some libs, like libblkid,
> don't handle multi-device filesystems or cloned devices with
> temp FSIDs very well. I'm aware of it.
> 
> I've been making some progress on fixing those cases, but it's
> a bit extensive since we first need enough test coverage,
> and recent reappear-device inline with that.
> 
> Let's see how we can support use cases with identical devices
> (where changing the UUID isn't an option) and keep things
> compatible with systemd and library tools.
> 

My current idea is to introduce a new ro compat flag, so that mounting 
that device will not go through the fsid lookup procedure completely.

But go through the common get_tree_bdev() routine, which will check if 
the fs is already mounted using bdev holder.
(And of course, no fsid recorded inside btrfs module)

This will make single device btrfs with that special flag to behave 
exactly like all the other filesystems.

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[Anand Jain]

On 14-Oct-25 9:57 AM, André Almeida wrote:
> Hi everyone,
> 
> When using overlayfs with the mount option index=on, the first time a directory is
> used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
> filesystem being used in the layers. If the upper dir is reused, overlayfs
> refuses to mount for a different filesystem, by comparing the UUID with what's
> stored at overlay.origin, and it fails with "failed to verify upper root origin"
> on dmesg. Remounting with the very same fs is supported and works fine.
> 
> However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
> disk image with btrfs, a random UUID is assigned for the following disks each
> time they are mounted, stored at temp_fsid and used across the kernel as the
> disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
> the original (and duplicated) UUID for all disks.
> 
> This feature doesn't work well with overlayfs with index=on, as when the image
> is mounted a second time, will get a different UUID and ovl will refuse to
> mount, breaking the user expectation that using the same image should work. A
> small script can be find in the end of this cover letter that illustrates this.
> 
>  From this, I can think of some options:
> 
> - Use statfs() internally to always get the fsid, that is persistent. The patch
> here illustrates that approach, but doesn't fully implement it.
> - Create a new sb op, called get_uuid() so the filesystem returns what's
> appropriated.
> - Have a workaround in ovl for btrfs.
> - Document this as unsupported, and userland needs to erase overlay.origin each
> time it wants to remount.
> - If ovl detects that temp_fsid and index are being used at the same time,
> refuses to mount.
> 
> I'm not sure which one would be better here, so I would like to hear some ideas
> on this.
> 
> Thanks!
> 	André
> 
> ---
> 
> To reproduce:
> 
> mkdir -p dir1 dir2
> 
> fallocate -l 300m ./disk1.img
> mkfs.btrfs -q -f ./disk1.img
> 
> # cloning the disks
> cp disk1.img disk2.img
> sudo mount -o loop ./disk1.img dir1
> sudo mount -o loop ./disk2.img dir2
> 
> mkdir -p dir2/lower aux/upper aux/work
> 
> # this works
> sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower
> 
> sudo umount dir2/lower
> sudo umount dir2
> 
> sudo mount -o loop ./disk2.img dir2

At this point, Btrfs assigns a new temporary FSID, but without it,
the test case fails.

Temp FSID support only came in with kernel v6.7, so wondering,
how is this test supposed to work on older kernels?

Thanks, Anand


> 
> # this doesn't works
> sudo mount -t overlay -o lowerdir=dir2/lower,upperdir=aux/upper,workdir=aux/work,userxattr none dir2/lower
> 
> André Almeida (1):
>    ovl: Use fsid as unique identifier for trusted origin
> 
>   fs/overlayfs/copy_up.c | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
> 

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[Amir Goldstein]

On Tue, Oct 14, 2025 at 3:57 AM André Almeida <andrealmeid@igalia.com> wrote:
>
> Hi everyone,
>
> When using overlayfs with the mount option index=on, the first time a directory is
> used as upper dir, overlayfs stores in a xattr "overlay.origin" the UUID of the
> filesystem being used in the layers. If the upper dir is reused, overlayfs
> refuses to mount for a different filesystem, by comparing the UUID with what's
> stored at overlay.origin, and it fails with "failed to verify upper root origin"
> on dmesg. Remounting with the very same fs is supported and works fine.
>
> However, btrfs mounts may have volatiles UUIDs. When mounting the exact same
> disk image with btrfs, a random UUID is assigned for the following disks each
> time they are mounted, stored at temp_fsid and used across the kernel as the
> disk UUID. `btrfs filesystem show` presents that. Calling statfs() however shows
> the original (and duplicated) UUID for all disks.
>
> This feature doesn't work well with overlayfs with index=on, as when the image
> is mounted a second time, will get a different UUID and ovl will refuse to
> mount, breaking the user expectation that using the same image should work. A
> small script can be find in the end of this cover letter that illustrates this.
>
> From this, I can think of some options:
>
> - Use statfs() internally to always get the fsid, that is persistent. The patch
> here illustrates that approach, but doesn't fully implement it.
> - Create a new sb op, called get_uuid() so the filesystem returns what's
> appropriated.

FWIW this operation already exists in export_operations.
It is currently only used by pnfs and only implemented by xfs.
I would nor object for overlayfs to use this method if implemented
and fall back to copying uuid directly from s_uuid
(better yet make it a vfs helper)
Note that commit
8f720d9f892e0 xfs: publish UUID in struct super_block
was done for a similar reason.
The xfs mount option nouuid is the poor man's solution for
mounting cloned disk images.

Thanks,
Amir.

Re: [RFC PATCH 0/1] ovl: brtfs' temp_fsid doesn't work with ovl index=on

[Christoph Hellwig]

On Wed, Oct 15, 2025 at 01:09:26PM +0200, Amir Goldstein wrote:
> FWIW this operation already exists in export_operations.
> It is currently only used by pnfs and only implemented by xfs.
> I would nor object for overlayfs to use this method if implemented
> and fall back to copying uuid directly from s_uuid

The get_uuid export operation is specifically about an on-disk superblock
that the pnfs client can match.  Overloading it for in-core information
would be very confusing and also problematic if it doesn't exist on-disk
in this format.  In retrospective it should be called get_disk_uuid or
something similar.