From mboxrd@z Thu Jan 1 00:00:00 1970 From: Oleg Nesterov Subject: Re: CGROUP =?utf-8?B?6rSA66CoIOusuOydmA==?= Date: Sun, 8 Sep 2013 18:00:07 +0200 Message-ID: <20130908160007.GA31903@redhat.com> References: <1286806.131871377667197297.JavaMail.weblogic@epv6ml01> <20130828134000.GA9295@htj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20130828134000.GA9295-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: containers-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: containers-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Tejun Heo Cc: =?utf-8?B?6rmA7J2A6riw?= , cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org SGkgVGVqdW4sCgpTb3JyeSBmb3IgZGVsYXksIHZhY2F0aW9uLgoKT24gMDgvMjgsIFRlanVuIEhl byB3cm90ZToKPgo+IEhleSwgb2xlZy4KPgo+IEV1bmtpIGlzIHJlcG9ydGluZyBhIHN0YWxsIGlu IHRoZSBmb2xsb3dpbmcgbG9vcCBpbgo+IGtlcm5lbC9jZ3JvdXAuYzo6Y2dyb3VwX2F0dGFjaF90 YXNrKCkKPgo+IE9uIFdlZCwgQXVnIDI4LCAyMDEzIGF0IDA1OjE5OjU3QU0gKzAwMDAsIOq5gOyd gOq4sCB3cm90ZToKPiA+Cj4gPiAgICAgIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQo+ID4gICAgICAgICBy Y3VfcmVhZF9sb2NrKCk7Cj4gPiAgICAgICAgIGRvIHsKPiA+ICAgICAgICAgICAgICAgICBzdHJ1 Y3QgdGFza19hbmRfY2dyb3VwIGVudDsKPiA+Cj4gPiAgICAgICAgICAgICAgICAgLyogQHRzayBl aXRoZXIgYWxyZWFkeSBleGl0ZWQgb3IgY2FuJ3QgZXhpdCB1bnRpbCB0aGUgZW5kICovCj4gPiAg ICAgICAgICAgICAgICAgaWYgKHRzay0+ZmxhZ3MgJiBQRl9FWElUSU5HKQo+ID4gICAgICAgICAg ICAgICAgICAgICAgICAgY29udGludWU7Cj4gPgo+ID4gICAgICAgICAgICAgICAgIC8qIGFzIHBl ciBhYm92ZSwgbnJfdGhyZWFkcyBtYXkgZGVjcmVhc2UsIGJ1dCBub3QgaW5jcmVhc2UuICovCj4g PiAgICAgICAgICAgICAgICAgQlVHX09OKGkgPj0gZ3JvdXBfc2l6ZSk7Cj4gPiAgICAgICAgICAg ICAgICAgZW50LnRhc2sgPSB0c2s7Cj4gPiAgICAgICAgICAgICAgICAgZW50LmNncnAgPSB0YXNr X2Nncm91cF9mcm9tX3Jvb3QodHNrLCByb290KTsKPiA+ICAgICAgICAgICAgICAgICAvKiBub3Ro aW5nIHRvIGRvIGlmIHRoaXMgdGFzayBpcyBhbHJlYWR5IGluIHRoZSBjZ3JvdXAgKi8KPiA+ICAg ICAgICAgICAgICAgICBpZiAoZW50LmNncnAgPT0gY2dycCkKPiA+ICAgICAgICAgICAgICAgICAg ICAgICAgIGNvbnRpbnVlOwo+ID4gICAgICAgICAgICAgICAgIC8qCj4gPiAgICAgICAgICAgICAg ICAgICogc2F5aW5nIEdGUF9BVE9NSUMgaGFzIG5vIGVmZmVjdCBoZXJlIGJlY2F1c2Ugd2UgZGlk IHByZWFsbG9jCj4gPiAgICAgICAgICAgICAgICAgICogZWFybGllciwgYnV0IGl0J3MgZ29vZCBm b3JtIHRvIGNvbW11bmljYXRlIG91ciBleHBlY3RhdGlvbnMuCj4gPiAgICAgICAgICAgICAgICAg ICovCj4gPiAgICAgICAgICAgICAgICAgcmV0dmFsID0gZmxleF9hcnJheV9wdXQoZ3JvdXAsIGks ICZlbnQsIEdGUF9BVE9NSUMpOwo+ID4gICAgICAgICAgICAgICAgIEJVR19PTihyZXR2YWwgIT0g MCk7Cj4gPiAgICAgICAgICAgICAgICAgaSsrOwo+ID4KPiA+ICAgICAgICAgICAgICAgICBpZiAo IXRocmVhZGdyb3VwKQo+ID4gICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWs7Cj4gPiAgICAg ICAgIH0gd2hpbGVfZWFjaF90aHJlYWQobGVhZGVyLCB0c2spOwo+ID4gLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4KPiB3aGVyZSB0aGUgaXRlcmF0aW9uIGdvZXMgbGlrZQo+ Cj4gICBsZWFkZXIgLT4gVGFzazEgLT4gVGFzazIgLT4gVGFzazMgIC0+IFRhc2sxCj4KPiBpZS4g bGVhZGVyIHNlZW1zIFJDVSB1bmxpbmtlZC4gIExvb2tpbmcgYXQgdGhlIHVzZXJzIG9mCj4gd2hp bGVfZWFjaF90aHJlYWQoKSwgSSdtIGNvbmZ1c2VkIGFib3V0IGl0cyBsb2NraW5nIHJlcXVpcmVt ZW50cy4KCkluIHNob3J0OiBpdCBpcyBicm9rZW4uIFRoaXMgd2FzIGFscmVhZHkgZGlzY3Vzc2Vk IHNldmVyYWwgdGltZXMgYnV0CmV2ZXJ5IHRpbWUgSSB3YXMgZGlzdHJhY3RlZC4KCkkgYWxyZWFk eSBoYXZlIHRoZSBwYXRjaGVzIHNvbWV3aGVyZSAocHJvYmFibHkgbm90IDEwMCUgZmluaXNoZWQp LAp3aWxsIHRyeSB0byByZXR1cm4gdG8gdGhpcyBwcm9ibGVtIHNvb24uCgpPbGVnLgoKX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KQ29udGFpbmVycyBtYWls aW5nIGxpc3QKQ29udGFpbmVyc0BsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xp c3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby9jb250YWluZXJz From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756950Ab3IHQGV (ORCPT ); Sun, 8 Sep 2013 12:06:21 -0400 Received: from mx1.redhat.com ([209.132.183.28]:3514 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754209Ab3IHQGT (ORCPT ); Sun, 8 Sep 2013 12:06:19 -0400 Date: Sun, 8 Sep 2013 18:00:07 +0200 From: Oleg Nesterov To: Tejun Heo Cc: =?utf-8?B?6rmA7J2A6riw?= , linux-kernel@vger.kernel.org, Li Zefan , containers@lists.linux-foundation.org, cgroups@vger.kernel.org Subject: Re: CGROUP =?utf-8?B?6rSA66CoIOusuOydmA==?= Message-ID: <20130908160007.GA31903@redhat.com> References: <1286806.131871377667197297.JavaMail.weblogic@epv6ml01> <20130828134000.GA9295@htj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20130828134000.GA9295@htj.dyndns.org> User-Agent: Mutt/1.5.18 (2008-05-17) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Tejun, Sorry for delay, vacation. On 08/28, Tejun Heo wrote: > > Hey, oleg. > > Eunki is reporting a stall in the following loop in > kernel/cgroup.c::cgroup_attach_task() > > On Wed, Aug 28, 2013 at 05:19:57AM +0000, 김은기 wrote: > > > > --------------------------------------------------------------------------- > > rcu_read_lock(); > > do { > > struct task_and_cgroup ent; > > > > /* @tsk either already exited or can't exit until the end */ > > if (tsk->flags & PF_EXITING) > > continue; > > > > /* as per above, nr_threads may decrease, but not increase. */ > > BUG_ON(i >= group_size); > > ent.task = tsk; > > ent.cgrp = task_cgroup_from_root(tsk, root); > > /* nothing to do if this task is already in the cgroup */ > > if (ent.cgrp == cgrp) > > continue; > > /* > > * saying GFP_ATOMIC has no effect here because we did prealloc > > * earlier, but it's good form to communicate our expectations. > > */ > > retval = flex_array_put(group, i, &ent, GFP_ATOMIC); > > BUG_ON(retval != 0); > > i++; > > > > if (!threadgroup) > > break; > > } while_each_thread(leader, tsk); > > --------------------------------------------------------------------------------------------- > > where the iteration goes like > > leader -> Task1 -> Task2 -> Task3 -> Task1 > > ie. leader seems RCU unlinked. Looking at the users of > while_each_thread(), I'm confused about its locking requirements. In short: it is broken. This was already discussed several times but every time I was distracted. I already have the patches somewhere (probably not 100% finished), will try to return to this problem soon. Oleg.