BUG: CGROUPS [2.6.28-rc6] Task migration does not clean reference to group

BUG: CGROUPS [2.6.28-rc6] Task migration does not clean reference to group

[sudhir kumar]

Hi,
I see an interesting bug in the cgroups. I found that cat /proc/cgroups
does not decrease the num_cgroups field when a group is deleted. It looks
to me that somewhere we miss to delete the reference, which leads to a failure
to unmount the cgroups filesystem.
The steps to produce the bug are:

mkdir /cpu
mount -t cgroup -ocpu cgroup /cpu
cd cpu
mkdir group1;
cd group1;
/./while &              # an infinite loop
echo pid > tasks        # pid is of infinite task
cd ..                   # we are now in root group
echo pid > /tasks       # move the task to root group
rmdir group1;           # Succeeds even when I was inside this group
in another shell
cd ..
umount /cpu;    # this fails for me as below
[root@malik-laptop /]# umount cpu/
umount: cpu/: device is busy

And I see that cat /proc/cgroups shows me still num_cgroups as 2;

I checked it on two kernels.
[root@malik-laptop cpu]# uname -a
Linux malik-laptop.in.ibm.com 2.6.26 #1 SMP Tue Aug 12 16:39:18 IST
2008 i686 i686 i386 GNU/Linux

[root@e325b sudhir]# uname -a
Linux e325b.in.ibm.com 2.6.28-rc6 #1 SMP Mon Dec 1 15:53:09 IST 2008
i686 athlon i386 GNU/Linux

A look on /proc/cgroups
root@e325b sudhir]# cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
#cpuset  0       1       1
#debug   0       1       1
#ns      0       1       1
#cpu     8       2       1
#cpuacct 0       1       1
#freezer 0       1       1
#

Please let me know if further information is required or I am doing
something wrong ?

Thanks
Sudhir

Re: BUG: CGROUPS [2.6.28-rc6] Task migration does not clean reference to group

[Li Zefan]

sudhir kumar wrote:
> Hi,
> I see an interesting bug in the cgroups. I found that cat /proc/cgroups
> does not decrease the num_cgroups field when a group is deleted. It looks
> to me that somewhere we miss to delete the reference, which leads to a failure
> to unmount the cgroups filesystem.

Ah, it's not a bug. :)

> The steps to produce the bug are:
> 
> mkdir /cpu
> mount -t cgroup -ocpu cgroup /cpu
> cd cpu
> mkdir group1;
> cd group1;
> /./while &              # an infinite loop

Seems whenever we create a background job, we'll have a pin to the directory where
the job is created. So though rmdir() removed the directory, the dentry's refcnt
is still >0, so cgroup_diput() in which number_of_cgroups is decreased is not called.
This explains why 'num_cgroups' showed in /proc/cgroups is not 1, and why umount failed.

You can try:
# mount -t ext3 /dev/sda6 /mnt
# cd /mnt
# sleep 10000 &
# cd ..
# umount /mnt
umount: /mnt: device is busy

And umount is OK for the below example:
# mount -t ext3 /dev/sda6 /mnt
# sleep 10000 &
# umount /mnt

> echo pid > tasks        # pid is of infinite task
> cd ..                   # we are now in root group
> echo pid > /tasks       # move the task to root group
> rmdir group1;           # Succeeds even when I was inside this group
> in another shell
> cd ..
> umount /cpu;    # this fails for me as below
> [root@malik-laptop /]# umount cpu/
> umount: cpu/: device is busy
> 
> And I see that cat /proc/cgroups shows me still num_cgroups as 2;
> 
> I checked it on two kernels.
> [root@malik-laptop cpu]# uname -a
> Linux malik-laptop.in.ibm.com 2.6.26 #1 SMP Tue Aug 12 16:39:18 IST
> 2008 i686 i686 i386 GNU/Linux
> 
> [root@e325b sudhir]# uname -a
> Linux e325b.in.ibm.com 2.6.28-rc6 #1 SMP Mon Dec 1 15:53:09 IST 2008
> i686 athlon i386 GNU/Linux
> 
> A look on /proc/cgroups
> root@e325b sudhir]# cat /proc/cgroups
> #subsys_name    hierarchy       num_cgroups     enabled
> #cpuset  0       1       1
> #debug   0       1       1
> #ns      0       1       1
> #cpu     8       2       1
> #cpuacct 0       1       1
> #freezer 0       1       1
> #
> 
> Please let me know if further information is required or I am doing
> something wrong ?
> 
> Thanks
> Sudhir

Re: BUG: CGROUPS [2.6.28-rc6] Task migration does not clean reference to group

[sudhir kumar]

On Tue, Dec 2, 2008 at 6:37 PM, Li Zefan <lizf@cn.fujitsu.com> wrote:
> sudhir kumar wrote:
>> Hi,
>> I see an interesting bug in the cgroups. I found that cat /proc/cgroups
>> does not decrease the num_cgroups field when a group is deleted. It looks
>> to me that somewhere we miss to delete the reference, which leads to a failure
>> to unmount the cgroups filesystem.
>
> Ah, it's not a bug. :)
Oh Sorry!
>
>> The steps to produce the bug are:
>>
>> mkdir /cpu
>> mount -t cgroup -ocpu cgroup /cpu
>> cd cpu
>> mkdir group1;
>> cd group1;
>> /./while &              # an infinite loop
>
> Seems whenever we create a background job, we'll have a pin to the directory where
> the job is created. So though rmdir() removed the directory, the dentry's refcnt
Thanks for that.
Should not rmdir fail in such a case ? Otherwise there is no other way
for the user than to kill his task and unmount the cgroup fs, in case
he needs to do so. Or shall we change the pin to point to the target
directory? No idea if I am demanding too much ;(


Thanks
Sudhir

Re: BUG: CGROUPS [2.6.28-rc6] Task migration does not clean reference to group

[Li Zefan]

sudhir kumar wrote:
> On Tue, Dec 2, 2008 at 6:37 PM, Li Zefan <lizf@cn.fujitsu.com> wrote:
>> sudhir kumar wrote:
>>> Hi,
>>> I see an interesting bug in the cgroups. I found that cat /proc/cgroups
>>> does not decrease the num_cgroups field when a group is deleted. It looks
>>> to me that somewhere we miss to delete the reference, which leads to a failure
>>> to unmount the cgroups filesystem.
>> Ah, it's not a bug. :)
> Oh Sorry!
>>> The steps to produce the bug are:
>>>
>>> mkdir /cpu
>>> mount -t cgroup -ocpu cgroup /cpu
>>> cd cpu
>>> mkdir group1;
>>> cd group1;
>>> /./while &              # an infinite loop
>> Seems whenever we create a background job, we'll have a pin to the directory where
>> the job is created. So though rmdir() removed the directory, the dentry's refcnt
> Thanks for that.
> Should not rmdir fail in such a case ? Otherwise there is no other way
> for the user than to kill his task and unmount the cgroup fs, in case
> he needs to do so. Or shall we change the pin to point to the target
> directory? No idea if I am demanding too much ;(
> 

You don't have to remove all the sub-dirs before umount cgroup fs. But if a process
has some pins to a directory, it's sure you can't umount the underlying fs, this
has nothing to do with cgroup.