[LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Avinesh Kumar]

Recently we started having failures where mounting cgroup in quick
succession after umount of same named hierarchy fails with EBUSY

cgroup_regression_test 1 TPASS: no kernel bug was found
mount: /tmp/LTP_cgroup_regression_test.ZFwS3JkX4c/cgroup: cgroup already mounted or mount point busy.
       dmesg(1) may have more information after failed mount system call.
cgroup_regression_test 2 TFAIL: Failed to mount cgroup filesystem

Updating test to use unique names for cgroup hierarchies in each
subtest.

Suggested-by: Michal Koutný <mkoutny@suse.com>
Signed-off-by: Avinesh Kumar <akumar@suse.de>
---
 .../controllers/cgroup/cgroup_regression_test.sh       | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh b/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
index 276231fe8..c1bf4810f 100755
--- a/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
+++ b/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
@@ -93,7 +93,7 @@ test1()
 	cgroup_regression_fork_processes &
 	sleep 1
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo1 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup filesystem"
 		kill -TERM $!
@@ -118,7 +118,7 @@ test2()
 	local val1
 	local val2
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo2 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "Failed to mount cgroup filesystem"
 		return
@@ -204,7 +204,7 @@ test4()
 		return
 	fi
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo4 cgroup cgroup/
 	mkdir cgroup/0
 	rmdir cgroup/0
 	tst_umount $PWD/cgroup
@@ -309,7 +309,7 @@ test_7_2()
 {
 	local subsys=$1
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo72 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup"
 		return
@@ -370,7 +370,7 @@ test7()
 #---------------------------------------------------------------------------
 test8()
 {
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo8 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup filesystem"
 		return
-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Avinesh Kumar]

I forgot to mention that I also tried to use
tst_umount() helper routine, but that did not help as umount does not
really report any failures here.

On Thursday, July 10, 2025 10:17:09 AM CEST Avinesh Kumar wrote:
> Recently we started having failures where mounting cgroup in quick
> succession after umount of same named hierarchy fails with EBUSY
> 
> cgroup_regression_test 1 TPASS: no kernel bug was found
> mount: /tmp/LTP_cgroup_regression_test.ZFwS3JkX4c/cgroup: cgroup already mounted or mount point busy.
>        dmesg(1) may have more information after failed mount system call.
> cgroup_regression_test 2 TFAIL: Failed to mount cgroup filesystem
> 
> Updating test to use unique names for cgroup hierarchies in each
> subtest.
> 
> Suggested-by: Michal Koutný <mkoutny@suse.com>
> Signed-off-by: Avinesh Kumar <akumar@suse.de>
> ---
>  .../controllers/cgroup/cgroup_regression_test.sh       | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh b/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
> index 276231fe8..c1bf4810f 100755
> --- a/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
> +++ b/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
> @@ -93,7 +93,7 @@ test1()
>  	cgroup_regression_fork_processes &
>  	sleep 1
>  
> -	mount -t cgroup -o none,name=foo cgroup cgroup/
> +	mount -t cgroup -o none,name=foo1 cgroup cgroup/
>  	if [ $? -ne 0 ]; then
>  		tst_res TFAIL "failed to mount cgroup filesystem"
>  		kill -TERM $!
> @@ -118,7 +118,7 @@ test2()
>  	local val1
>  	local val2
>  
> -	mount -t cgroup -o none,name=foo cgroup cgroup/
> +	mount -t cgroup -o none,name=foo2 cgroup cgroup/
>  	if [ $? -ne 0 ]; then
>  		tst_res TFAIL "Failed to mount cgroup filesystem"
>  		return
> @@ -204,7 +204,7 @@ test4()
>  		return
>  	fi
>  
> -	mount -t cgroup -o none,name=foo cgroup cgroup/
> +	mount -t cgroup -o none,name=foo4 cgroup cgroup/
>  	mkdir cgroup/0
>  	rmdir cgroup/0
>  	tst_umount $PWD/cgroup
> @@ -309,7 +309,7 @@ test_7_2()
>  {
>  	local subsys=$1
>  
> -	mount -t cgroup -o none,name=foo cgroup cgroup/
> +	mount -t cgroup -o none,name=foo72 cgroup cgroup/
>  	if [ $? -ne 0 ]; then
>  		tst_res TFAIL "failed to mount cgroup"
>  		return
> @@ -370,7 +370,7 @@ test7()
>  #---------------------------------------------------------------------------
>  test8()
>  {
> -	mount -t cgroup -o none,name=foo cgroup cgroup/
> +	mount -t cgroup -o none,name=foo8 cgroup cgroup/
>  	if [ $? -ne 0 ]; then
>  		tst_res TFAIL "failed to mount cgroup filesystem"
>  		return
> 





-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Andrea Cervesato via ltp]

On 7/10/25 10:24 AM, Avinesh Kumar wrote:
> I forgot to mention that I also tried to use
> tst_umount() helper routine, but that did not help as umount does not
> really report any failures here.

The solution doesn't really convince me. If it's true that a util-linux 
upgrade to v2.41 is causing this issue, we should probably research why 
this is happening in there and (eventually) fix it. Or, if it's not a 
library bug, we should fix tst_umount() in order to recognize when a 
cgroups folder hasn't been fully unmounted.

@Cyril WDYT ?

- Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Cyril Hrubis]

Hi!
> > I forgot to mention that I also tried to use
> > tst_umount() helper routine, but that did not help as umount does not
> > really report any failures here.
> 
> The solution doesn't really convince me. If it's true that a util-linux 
> upgrade to v2.41 is causing this issue, we should probably research why 
> this is happening in there and (eventually) fix it. Or, if it's not a 
> library bug, we should fix tst_umount() in order to recognize when a 
> cgroups folder hasn't been fully unmounted.
> 
> @Cyril WDYT ?

This indeed looks like a workaround for a bug either in the test or in
the system.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Andrea Cervesato via ltp]

On 7/10/25 11:00 AM, Cyril Hrubis wrote:
> Hi!
>>> I forgot to mention that I also tried to use
>>> tst_umount() helper routine, but that did not help as umount does not
>>> really report any failures here.
>> The solution doesn't really convince me. If it's true that a util-linux
>> upgrade to v2.41 is causing this issue, we should probably research why
>> this is happening in there and (eventually) fix it. Or, if it's not a
>> library bug, we should fix tst_umount() in order to recognize when a
>> cgroups folder hasn't been fully unmounted.
>>
>> @Cyril WDYT ?
> This indeed looks like a workaround for a bug either in the test or in
> the system.
>
Perhaps, by placing sync before every new test which is going to mount a 
new cgroup folder we fix the issue as well.
I'm wondering if we should dig into it a big deeper...

- Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Andrea Cervesato via ltp]

On 7/11/25 2:08 PM, Michal Koutný wrote:
> On Fri, Jul 11, 2025 at 12:36:15PM +0200, Andrea Cervesato <andrea.cervesato@suse.com> wrote:
>> On 7/10/25 11:00 AM, Cyril Hrubis wrote:
> ...
>>> This indeed looks like a workaround for a bug either in the test or in
>>> the system.
>>>
>> Perhaps, by placing sync before every new test which is going to mount a new
>> cgroup folder we fix the issue as well.
> That may only a workaround too. My bet is you'd rather need some RCU
> barriers or workqueue flushing, thus the simple separation by name. The
> cleanup path is just so asynchronous (the code had already attempted
> some nastiness [1], however, I'm suspecting its efficiency may be
> reduced under layers of VFS reworks since then).
>
>> I'm wondering if we should dig into it a big deeper...
> It may be worth checking that there's a change in util-linux that has
> (presumably) affected timings.
> Note that unmounting a cgroup hierarchy is operation that typically
> happens once during "lifetime" (if at all) -- so it's your call.
I see, thanks for the suggestion. I will take a look.
>
> HTH,
> Michal
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=970317aa48c6ef66cd023c039c2650c897bad927

This git commit is part of cgroup v1 code and now in 
kernel/cgroup/cgroup-v1.c:cgroup1_root_to_use, but I don't see the same 
mechanism used by cgroup v2, which is our main version. Is that 
something which might affect our system?

- Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Andrea Cervesato via ltp]

On 7/11/25 3:15 PM, Michal Koutný wrote:
> On Fri, Jul 11, 2025 at 02:41:18PM +0200, Andrea Cervesato <andrea.cervesato@suse.com> wrote:
>> This git commit is part of cgroup v1 code and now in
>> kernel/cgroup/cgroup-v1.c:cgroup1_root_to_use, but I don't see the same
>> mechanism used by cgroup v2, which is our main version.
> The patched tests use cgroup v1.
>
>> Is that something which might affect our system?
> Since there's only one cgroup v2 (unified) hierarchy, it sort of lives
> forever (statically alloc'd), so I wouldn't expect such residuals on v2
> (with the current implementation).
>
> Michal

I bisected util-linux, but it doesn't seem to be the problem. On my 
system (Tumbleweed), we are facing the same failure, no matter the 
commit that is tested in util-linux between 2.39 and 2.41 versions.

I guess this issue might be caused by a specific setup of the Linux 
kernel and its async nature, mixed with some services which are slowing 
down the umount process.

I'm wondering how to proceed, since this is actually a soft-failure and 
any fix to the test might hide future bugs.

- Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Avinesh Kumar]

Hi all¸

I would like to ask if there are any other suggestions on how to address this
issue.
I tried a rather simpler approach (as suggested by Martin) of just adding a
delay using sleep after the umount call, which also seems to solve this issue
of incomplete unmounting.

--- a/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
+++ b/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
@@ -102,10 +102,11 @@ test1()
        cat cgroup/tasks > /dev/null
 
        kill -TERM $!
        wait $! 2>/dev/null
        umount cgroup/
+       sleep 2
        check_kernel_bug
 }

I will send this in the patch if we agree to use this approach instead of
unique names.

Thanks,
Avinesh

On Monday, July 14, 2025 1:56:51 PM CEST Andrea Cervesato wrote:
> On 7/11/25 3:15 PM, Michal Koutný wrote:
> > On Fri, Jul 11, 2025 at 02:41:18PM +0200, Andrea Cervesato <andrea.cervesato@suse.com> wrote:
> >> This git commit is part of cgroup v1 code and now in
> >> kernel/cgroup/cgroup-v1.c:cgroup1_root_to_use, but I don't see the same
> >> mechanism used by cgroup v2, which is our main version.
> > The patched tests use cgroup v1.
> >
> >> Is that something which might affect our system?
> > Since there's only one cgroup v2 (unified) hierarchy, it sort of lives
> > forever (statically alloc'd), so I wouldn't expect such residuals on v2
> > (with the current implementation).
> >
> > Michal
> 
> I bisected util-linux, but it doesn't seem to be the problem. On my 
> system (Tumbleweed), we are facing the same failure, no matter the 
> commit that is tested in util-linux between 2.39 and 2.41 versions.
> 
> I guess this issue might be caused by a specific setup of the Linux 
> kernel and its async nature, mixed with some services which are slowing 
> down the umount process.
> 
> I'm wondering how to proceed, since this is actually a soft-failure and 
> any fix to the test might hide future bugs.
> 
> - Andrea
> 
> 






-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Martin Doucha]

On 30. 07. 25 13:32, Avinesh Kumar wrote:
> Hi all¸
> 
> I would like to ask if there are any other suggestions on how to address this
> issue.
> I tried a rather simpler approach (as suggested by Martin) of just adding a
> delay using sleep after the umount call, which also seems to solve this issue
> of incomplete unmounting.
> 
> --- a/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
> +++ b/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh
> @@ -102,10 +102,11 @@ test1()
>          cat cgroup/tasks > /dev/null
>   
>          kill -TERM $!
>          wait $! 2>/dev/null
>          umount cgroup/
> +       sleep 2
>          check_kernel_bug
>   }
> 
> I will send this in the patch if we agree to use this approach instead of
> unique names.

Well, this looks like the unmount happens asynchronously and the command 
exits before it's actually finished. In that case, sync would be the 
best solution. But I think we need some feedback from kernel developers 
whether the asynchronous behavior is intentional or not. Should the 
umount command wait for the task to finish?

-- 
Martin Doucha   mdoucha@suse.cz
SW Quality Engineer
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] cgroup_regression_test: Use unique names for hierarchies in each test

[Michal Koutný via ltp]

On Mon, Jul 14, 2025 at 01:56:51PM +0200, Andrea Cervesato <andrea.cervesato@suse.com> wrote:
> I'm wondering how to proceed, since this is actually a soft-failure and any
> fix to the test might hide future bugs.

You can take the (unique names) fix from Avinesh and add a dedicated
test for the unmounting (and solve (soft)failure only in that).

0.02€,
Michal

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp