[RFC] ceph: strange mount/unmount behavior

[Viacheslav Dubeyko <Slava.Dubeyko@ibm.com>]

Hello,

I am investigating an issue with generic/604:

sudo ./check generic/604
FSTYP         -- ceph
PLATFORM      -- Linux/x86_64 ceph-0005 6.17.0-rc1+ #29 SMP PREEMPT_DYNAMIC Mon
Aug 25 13:06:10 PDT 2025
MKFS_OPTIONS  -- 192.168.1.213:6789:/scratch
MOUNT_OPTIONS -- -o name=admin 192.168.1.213:6789:/scratch /mnt/cephfs/scratch

generic/604 10s ... - output mismatch (see
XFSTESTS/xfstestsdev/results//generic/604.out.bad)
    --- tests/generic/604.out	2025-02-25 13:05:32.515668548 -0800
    +++ XFSTESTS/xfstests-dev/results//generic/604.out.bad	2025-08-25
14:25:49.256780397 -0700
    @@ -1,2 +1,3 @@
     QA output created by 604
    +umount: /mnt/cephfs/scratch: target is busy.
     Silence is golden
    ...
    (Run 'diff -u XFSTESTS/xfstests-dev/tests/generic/604.out XFSTESTS/xfstests-
dev/results//generic/604.out.bad'  to see the entire diff)
Ran: generic/604
Failures: generic/604
Failed 1 of 1 tests

As far as I can see, the generic/604 intentionally delays the unmount and mount
operation starts before unmount finish:

# For overlayfs, avoid unmounting the base fs after _scratch_mount tries to
# mount the base fs.  Delay the mount attempt by a small amount in the hope
# that the mount() call will try to lock s_umount /after/ umount has already
# taken it.
$UMOUNT_PROG $SCRATCH_MNT &
sleep 0.01s ; _scratch_mount
wait

As a result, we have this issue because a mnt_count is bigger than expected one
in propagate_mount_busy() [1]:

	} else {
		smp_mb(); // paired with __legitimize_mnt()
		shrink_submounts(mnt);
		retval = -EBUSY;
		if (!propagate_mount_busy(mnt, 2)) {
			umount_tree(mnt, UMOUNT_PROPAGATE|UMOUNT_SYNC);
			retval = 0;
		}
	}


[   71.347372] pid 3762 do_umount():2022 finished:  mnt_get_count(mnt) 3

But if I am trying to understand what is going on during mount, then I can see
that I can mount the same file system instance multiple times even for the same
mount point:

192.168.1.195:6789,192.168.1.212:6789,192.168.1.213:6789:/ on /mnt/cephfs type
ceph (rw,relatime,name=admin,secret=<hidden>,fsid=31977b06-8cdb-42a9-97ad-
d6a7d59a42dd,acl,mds_namespace=cephfs)
192.168.1.195:6789,192.168.1.212:6789,192.168.1.213:6789:/ on /mnt/TestCephFS
type ceph (rw,relatime,name=admin,secret=<hidden>,fsid=31977b06-8cdb-42a9-97ad-
d6a7d59a42dd,acl,mds_namespace=cephfs)
192.168.1.195:6789,192.168.1.212:6789,192.168.1.213:6789:/ on /mnt/cephfs type
ceph (rw,relatime,name=admin,secret=<hidden>,fsid=31977b06-8cdb-42a9-97ad-
d6a7d59a42dd,acl,mds_namespace=cephfs)
192.168.1.195:6789,192.168.1.212:6789,192.168.1.213:6789:/ on /mnt/cephfs type
ceph (rw,relatime,name=admin,secret=<hidden>,fsid=31977b06-8cdb-42a9-97ad-
d6a7d59a42dd,acl,mds_namespace=cephfs)
192.168.1.195:6789,192.168.1.212:6789,192.168.1.213:6789:/ on /mnt/cephfs type
ceph (rw,relatime,name=admin,secret=<hidden>,fsid=31977b06-8cdb-42a9-97ad-
d6a7d59a42dd,acl,mds_namespace=cephfs)

And it looks really confusing to me. OK, let's imagine that mounting the same
file system instance into different folders (for example, /mnt/TestCephFS and
/mnt/cephfs) could make sense. However, I am not sure that it is correct
behavior. But mounting the same file system instance into the same folder
doesn't make sense to me. Maybe, I am missing something important here.

Am I correct here? Is it expected behavior? I assume that CephFS has incorrect
mount logic that creates the issue during umount operation? Any thoughts?

Thanks,
Slava.

[1] https://elixir.bootlin.com/linux/v6.17-rc1/source/fs/namespace.c#L2002