generic/645 failing on ext4, xfs (probably others) on all LTS kernels

generic/645 failing on ext4, xfs (probably others) on all LTS kernels

[Theodore Ts'o]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

The test generic/645 is failing on (at least) 6.6, 6.1, 5.15 LTS
kernels.

This fix is apparently commit dacfd001eaf2 ("fs/mnt_idmapping.c:
Return -EINVAL when no map is written"), but in order to take this
patch, it looks like we need to backport the 4 patch series
"mnt_idmapping: decouple from namespaces"[1] (and possibly others; I
haven't tried yet).

[1] https://lore.kernel.org/all/20231122-vfs-mnt_idmap-v1-0-dae4abdde5bd@kernel.org/

This looks fairly involved so the questions I have are:

(1) Should we request this patch series plus commit dacfd001eaf2 into
the stable kernels --- or should I just add a versioned excludes[2]
and just skip generic/645 from all kernels older than Linux 6.9 if we
think it's too involved and/or risky to backport these id mapping
changes?

(2) How much do we care that generic/645 is failing on LTS kernels?
Are user/applications going to notice or care?

Thanks,

					- Ted

[2] Like this:

diff --git a/test-appliance/files/root/fs/global_exclude b/test-appliance/files/root/fs/global_exclude
index d7acf89f..42902152 100644
--- a/test-appliance/files/root/fs/global_exclude
+++ b/test-appliance/files/root/fs/global_exclude
@@ -30,6 +30,14 @@ generic/484
 generic/554
 #endif
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6,9,0)
+// This test failure is fixed by commit dacfd001eaf2
+// ("fs/mnt_idmapping.c: Return -EINVAL when no map is written"),
+// but it's too involved to backport it and its dependencies to
+// the LTS kernels.
+generic/645
+#endif
+
 #ifndef IS_DAX_CONFIG
 // Unless we are testing the dax config, we can exclude all dax tests
 -g dax

[-- Attachment #2: Type: message/rfc822, Size: 23364 bytes --]

From: Xfstests Reporter <tytso@thunk.org>
To: theodore.tso@gmail.com
Subject: xfstests results ltm-20241016165408-0014 6.6.61-rc1-xfstests-g4744f45e99fb-8
Date: Sun, 10 Nov 2024 15:12:05 +0000 (UTC)
Message-ID: <kF4Z6m8lS4qxluY2wPzMCA@geopod-ismtpd-6>

TESTRUNID: ltm-20241016165408-0014
KERNEL:    kernel 6.6.61-rc1-xfstests-g4744f45e99fb #8 SMP PREEMPT_DYNAMIC Sun Nov 10 05:14:53 EST 2024 x86_64
CMDLINE:   -c ext4/all,xfs/all -g auto --repo stable-rc.git --watch linux-6.6.y
CPUS:      2
MEM:       7680

ext4/4k: 576 tests, 2 failures, 55 skipped, 6031 seconds
  Failures: generic/365 generic/645
ext4/1k: 570 tests, 3 failures, 59 skipped, 5818 seconds
  Failures: generic/627 generic/645 generic/750
ext4/ext3: 568 tests, 2 failures, 148 skipped, 4243 seconds
  Failures: generic/365 generic/645
ext4/encrypt: 551 tests, 2 failures, 171 skipped, 4186 seconds
  Failures: generic/365 generic/645
ext4/nojournal: 568 tests, 2 failures, 126 skipped, 4458 seconds
  Failures: generic/365 generic/645
ext4/ext3conv: 573 tests, 3 failures, 57 skipped, 6299 seconds
  Failures: generic/347 generic/365 generic/645
ext4/adv: 569 tests, 2 failures, 63 skipped, 4802 seconds
  Failures: generic/365 generic/645
ext4/dioread_nolock: 574 tests, 2 failures, 55 skipped, 4746 seconds
  Failures: generic/365 generic/645
ext4/data_journal: 569 tests, 2 failures, 132 skipped, 4243 seconds
  Failures: generic/365 generic/645
ext4/bigalloc_4k: 547 tests, 2 failures, 58 skipped, 4770 seconds
  Failures: generic/365 generic/645
ext4/bigalloc_1k: 548 tests, 1 failures, 69 skipped, 4855 seconds
  Failures: generic/645
ext4/dax: 562 tests, 2 failures, 159 skipped, 2977 seconds
  Failures: generic/365 generic/645
xfs/4k: 1157 tests, 13 failures, 181 skipped, 11983 seconds
  Failures: generic/091 generic/127 generic/263 generic/363 generic/365 
    generic/616 generic/645 xfs/629 xfs/630 xfs/631 xfs/632 xfs/806
  Flaky: generic/455: 20% (1/5)
xfs/1k: 1157 tests, 12 failures, 169 skipped, 12621 seconds
  Failures: generic/091 generic/127 generic/263 generic/363 generic/616 
    generic/645 xfs/629 xfs/630 xfs/631 xfs/632 xfs/806
  Flaky: generic/560: 20% (1/5)
xfs/v4: 1156 tests, 11 failures, 476 skipped, 8452 seconds
  Failures: generic/363 generic/365 generic/645 xfs/348 xfs/629 xfs/630 
    xfs/631 xfs/632 xfs/803 xfs/804 xfs/806
xfs/adv: 1157 tests, 13 failures, 167 skipped, 11341 seconds
  Failures: generic/091 generic/127 generic/263 generic/363 generic/365 
    generic/616 generic/645 xfs/157 xfs/629 xfs/630 xfs/631 xfs/632 
    xfs/806
xfs/quota: 1157 tests, 12 failures, 166 skipped, 12417 seconds
  Failures: generic/091 generic/127 generic/263 generic/363 generic/365 
    generic/616 generic/645 xfs/629 xfs/630 xfs/631 xfs/632 xfs/806
xfs/quota_1k: 1157 tests, 13 failures, 169 skipped, 14282 seconds
  Failures: generic/091 generic/127 generic/263 generic/363 generic/627 
    generic/645 xfs/629 xfs/630 xfs/631 xfs/632 xfs/806
  Flaky: generic/112: 20% (1/5)   generic/616: 80% (4/5)
xfs/dirblock_8k: 1157 tests, 13 failures, 166 skipped, 12183 seconds
  Failures: generic/091 generic/127 generic/263 generic/363 generic/365 
    generic/616 generic/645 xfs/629 xfs/630 xfs/631 xfs/632 xfs/806
  Flaky: generic/075: 80% (4/5)
xfs/realtime: 1156 tests, 6 failures, 521 skipped, 9530 seconds
  Failures: generic/363 generic/365 generic/455 generic/645 xfs/629 xfs/806
xfs/realtime_28k_logdev: 1156 tests, 5 failures, 602 skipped, 16344 seconds
  Failures: generic/363 generic/365 generic/645 xfs/598 xfs/806
xfs/realtime_logdev: 1156 tests, 7 failures, 575 skipped, 12594 seconds
  Failures: generic/363 generic/365 generic/645 xfs/598 xfs/629 xfs/789 
    xfs/806
xfs/logdev: 1157 tests, 14 failures, 236 skipped, 15071 seconds
  Failures: generic/091 generic/127 generic/263 generic/363 generic/365 
    generic/616 generic/645 xfs/598 xfs/629 xfs/630 xfs/631 xfs/632 
    xfs/806
  Flaky: generic/112: 60% (3/5)
xfs/dax: 1167 tests, 7 failures, 552 skipped, 5763 seconds
  Failures: generic/365 generic/645 xfs/629 xfs/630 xfs/631 xfs/632 xfs/806
Totals: 21269 tests, 5132 skipped, 739 failures, 0 errors, 186798s

FSTESTIMG: gce-xfstests/xfstests-amd64-202410151341
FSTESTPRJ: gce-xfstests
FSTESTVER: blktests f043065 (Thu, 20 Jun 2024 16:26:22 +0900)
FSTESTVER: fio  fio-3.38 (Wed, 2 Oct 2024 12:53:13 -0600)
FSTESTVER: fsverity v1.6 (Wed, 20 Mar 2024 21:21:46 -0700)
FSTESTVER: ima-evm-utils v1.5 (Mon, 6 Mar 2023 07:40:07 -0500)
FSTESTVER: libaio   libaio-0.3.108-82-gb8eadc9 (Thu, 2 Jun 2022 13:33:11 +0200)
FSTESTVER: ltp  20240930 (Mon, 30 Sep 2024 12:27:12 +0200)
FSTESTVER: quota  v4.05-69-g68952f1 (Mon, 7 Oct 2024 15:45:56 -0400)
FSTESTVER: util-linux v2.40.2 (Thu, 4 Jul 2024 09:59:17 +0200)
FSTESTVER: xfsprogs v6.10.1-91-g42523142 (Tue, 8 Oct 2024 14:31:31 +0200)
FSTESTVER: xfstests-bld 71bcf39c (Tue, 15 Oct 2024 13:27:36 -0400)
FSTESTVER: xfstests v2024.10.14-11-g173cdbc07 (Tue, 15 Oct 2024 09:31:29 -0400)
FSTESTVER: zz_build-distro bookworm
FSTESTSET: -g auto
FSTESTOPT: aex

Re: generic/645 failing on ext4, xfs (probably others) on all LTS kernels

[Matthew Wilcox]

On Sun, Nov 10, 2024 at 01:05:33PM -0500, Theodore Ts'o wrote:
> (2) How much do we care that generic/645 is failing on LTS kernels?
> Are user/applications going to notice or care?

... if userspace applications won't notice this breakage, then why do we
run this test?

Re: generic/645 failing on ext4, xfs (probably others) on all LTS kernels

[Christian Brauner]

On Sun, Nov 10, 2024 at 01:05:33PM -0500, Theodore Ts'o wrote:
> The test generic/645 is failing on (at least) 6.6, 6.1, 5.15 LTS
> kernels.
> 
> This fix is apparently commit dacfd001eaf2 ("fs/mnt_idmapping.c:
> Return -EINVAL when no map is written"), but in order to take this
> patch, it looks like we need to backport the 4 patch series
> "mnt_idmapping: decouple from namespaces"[1] (and possibly others; I
> haven't tried yet).
> 
> [1] https://lore.kernel.org/all/20231122-vfs-mnt_idmap-v1-0-dae4abdde5bd@kernel.org/
> 
> This looks fairly involved so the questions I have are:
> 
> (1) Should we request this patch series plus commit dacfd001eaf2 into
> the stable kernels --- or should I just add a versioned excludes[2]
> and just skip generic/645 from all kernels older than Linux 6.9 if we
> think it's too involved and/or risky to backport these id mapping
> changes?
> 
> (2) How much do we care that generic/645 is failing on LTS kernels?
> Are user/applications going to notice or care?

No userspace used an empty idmapping and it was unclear whether the
behavior would be well-specified so the patch changed that quite some
time ago.

Backporting this to older LTS kernels isn't difficult. We just need
custom patches for the LTS kernels but they should all be very simple.

Alternatively, you can just ignore the test on older kernels.

Re: generic/645 failing on ext4, xfs (probably others) on all LTS kernels

[Theodore Ts'o]

On Mon, Nov 11, 2024 at 09:52:07AM +0100, Christian Brauner wrote:

> behavior would be well-specified so the patch changed that quite some
> time ago.
> 
> Backporting this to older LTS kernels isn't difficult. We just need
> custom patches for the LTS kernels but they should all be very simple.
> 
> Alternatively, you can just ignore the test on older kernels.

Well, what the custom patch to look like wasn't obvious to me, but
that's because I'm not sufficiently familiar with the id mapping code.

So I'll just ignore the test on older kernels.  If someone wants to
create the custom patch, I'll revert the versioned exclude for
{kvm,gce}-xfsteests.

Thanks,

						- Ted

Re: generic/645 failing on ext4, xfs (probably others) on all LTS kernels

[Ojaswin Mujoo]

On Fri, Nov 15, 2024 at 08:34:07AM -0500, Theodore Ts'o wrote:
> On Mon, Nov 11, 2024 at 09:52:07AM +0100, Christian Brauner wrote:
> 
> > behavior would be well-specified so the patch changed that quite some
> > time ago.
> > 
> > Backporting this to older LTS kernels isn't difficult. We just need
> > custom patches for the LTS kernels but they should all be very simple.
> > 
> > Alternatively, you can just ignore the test on older kernels.
> 
> Well, what the custom patch to look like wasn't obvious to me, but
> that's because I'm not sufficiently familiar with the id mapping code.
> 
> So I'll just ignore the test on older kernels.  If someone wants to
> create the custom patch, I'll revert the versioned exclude for
> {kvm,gce}-xfsteests.
> 
> Thanks,
> 
> 						- Ted
> 

Hi,

We did notice this as well during our internal testing. I'm unfamiliar
with the code but after spending some time I came up with the
following patch that did fix it for us (it's a rough patch, might not
apply cleanly to any stable kernel). Again, I'm not 100% sure if this is
the right place to patch it but if it looks good to peopel more familiar with
the area then I can try to send the backports to the stable.

---
 fs/namespace.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/fs/namespace.c b/fs/namespace.c
index 7906c5327f4f..368a60e48861 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -3995,6 +3995,12 @@ static int can_idmap_mount(const struct mount_kattr *kattr, struct mount *mnt)
 	if (!kattr->mnt_idmap)
 		return 0;
 
+	/* Don't allow idmaps with no mapping defined */
+	if (kattr->mnt_userns->uid_map.nr_extents == 0 ||
+	    kattr->mnt_userns->gid_map.nr_extents == 0)
+		return -EINVAL;
+
+
 	/*
 	 * Creating an idmapped mount with the filesystem wide idmapping
 	 * doesn't make sense so block that. We don't allow mushy semantics.
-- 
2.43.0

Regards,
ojaswin