cpu shielding.

cpu shielding.

[atp]

Hello,

  Apologies if I'm about to ask a frequently asked question - I did
check back over the last couple of months. 

  Is anyone working on cpu shielding for processes inside a cpu cgroup? 

  We would like to run Java in containers, and unfortunately it likes to
know how many processors there are in the system - to initialise thread
pools and such like. 

  I was thinking along these lines;

--- fs/proc/stat.c.orig	2010-05-21 11:32:32.941258466 +0000
+++ fs/proc/stat.c	2010-05-21 11:40:47.681259133 +0000
@@ -39,7 +39,9 @@
 	getboottime(&boottime);
 	jif = boottime.tv_sec;
 
-	for_each_possible_cpu(i) {
+//	for_each_possible_cpu(i) {
+//	// refer to the visible cpus.
+	for_each_cpu_and(i,cpu_possible_mask,(&current->cpus_allowed)) {
 		user = cputime64_add(user, kstat_cpu(i).cpustat.user);
 		nice = cputime64_add(nice, kstat_cpu(i).cpustat.nice);
 		system = cputime64_add(system, kstat_cpu(i).cpustat.system);
@@ -78,7 +80,10 @@
 		(unsigned long long)cputime64_to_clock_t(steal),
 		(unsigned long long)cputime64_to_clock_t(guest),
 		(unsigned long long)cputime64_to_clock_t(guest_nice));
-	for_each_online_cpu(i) {
+     
+//	for_each_online_cpu(i) {
+//	// cgroup. 
+	for_each_cpu_and(i,cpu_online_mask,(&current->cpus_allowed)) {
 
 		/* Copy values here to work around gcc-2.95.3, gcc-2.96 */
 		user = kstat_cpu(i).cpustat.user;

  I'm sure that there are nicer ways of doing this, but Serge Hallyn
suggested a while ago that I post here. Even though it says 2010, the
patch above looks like it will go against 3.2.4 ok. 
  
  Thanks,
     Andy

-- 
Andrew Phillips
Head of Systems

Direct: +44 (0)203 192 2509
Mobile: +44 (0)7595 242 900

LMAX, Yellow Building, 1A Nicholas Road,  London, W11 4AN



The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company.

Re: cpu shielding.

[Serge Hallyn]

Quoting atp (Andrew.Phillips-xheW4WVAX9Y@public.gmane.org):
> Hello,
> 
>   Apologies if I'm about to ask a frequently asked question - I did
> check back over the last couple of months. 
> 
>   Is anyone working on cpu shielding for processes inside a cpu cgroup? 
> 
>   We would like to run Java in containers, and unfortunately it likes to
> know how many processors there are in the system - to initialise thread
> pools and such like. 
> 
>   I was thinking along these lines;
> 
> --- fs/proc/stat.c.orig	2010-05-21 11:32:32.941258466 +0000
> +++ fs/proc/stat.c	2010-05-21 11:40:47.681259133 +0000
> @@ -39,7 +39,9 @@
>  	getboottime(&boottime);
>  	jif = boottime.tv_sec;
>  
> -	for_each_possible_cpu(i) {
> +//	for_each_possible_cpu(i) {
> +//	// refer to the visible cpus.
> +	for_each_cpu_and(i,cpu_possible_mask,(&current->cpus_allowed)) {
>  		user = cputime64_add(user, kstat_cpu(i).cpustat.user);
>  		nice = cputime64_add(nice, kstat_cpu(i).cpustat.nice);
>  		system = cputime64_add(system, kstat_cpu(i).cpustat.system);
> @@ -78,7 +80,10 @@
>  		(unsigned long long)cputime64_to_clock_t(steal),
>  		(unsigned long long)cputime64_to_clock_t(guest),
>  		(unsigned long long)cputime64_to_clock_t(guest_nice));
> -	for_each_online_cpu(i) {
> +     
> +//	for_each_online_cpu(i) {
> +//	// cgroup. 
> +	for_each_cpu_and(i,cpu_online_mask,(&current->cpus_allowed)) {
>  
>  		/* Copy values here to work around gcc-2.95.3, gcc-2.96 */
>  		user = kstat_cpu(i).cpustat.user;
> 
>   I'm sure that there are nicer ways of doing this, but Serge Hallyn
> suggested a while ago that I post here. Even though it says 2010, the
> patch above looks like it will go against 3.2.4 ok. 
>   
>   Thanks,
>      Andy

I'm afraid I haven't been following recent upstream discussions on
this, but there are other people who want proc to show cgroup-limited
information.  See for instance http://lwn.net/Articles/460310/ .  Glauber
has also brought this up since then.  I'd recommend pinging him.

I'm all for /proc showing cgroup-filtered information, unconditionally.

-serge

Re: cpu shielding.

[Zhu Yanhai]

2012/2/6 atp <Andrew.Phillips@lmax.com>:
> Hello,
>
>  Apologies if I'm about to ask a frequently asked question - I did
> check back over the last couple of months.
>
>  Is anyone working on cpu shielding for processes inside a cpu cgroup?
>
>  We would like to run Java in containers, and unfortunately it likes to
> know how many processors there are in the system - to initialise thread
> pools and such like.
>
>  I was thinking along these lines;
>
> --- fs/proc/stat.c.orig 2010-05-21 11:32:32.941258466 +0000
> +++ fs/proc/stat.c      2010-05-21 11:40:47.681259133 +0000
> @@ -39,7 +39,9 @@
>        getboottime(&boottime);
>        jif = boottime.tv_sec;
>
> -       for_each_possible_cpu(i) {
> +//     for_each_possible_cpu(i) {
> +//     // refer to the visible cpus.
> +       for_each_cpu_and(i,cpu_possible_mask,(&current->cpus_allowed)) {
>                user = cputime64_add(user, kstat_cpu(i).cpustat.user);
>                nice = cputime64_add(nice, kstat_cpu(i).cpustat.nice);
>                system = cputime64_add(system, kstat_cpu(i).cpustat.system);
> @@ -78,7 +80,10 @@
>                (unsigned long long)cputime64_to_clock_t(steal),
>                (unsigned long long)cputime64_to_clock_t(guest),
>                (unsigned long long)cputime64_to_clock_t(guest_nice));
> -       for_each_online_cpu(i) {
> +
> +//     for_each_online_cpu(i) {
> +//     // cgroup.
> +       for_each_cpu_and(i,cpu_online_mask,(&current->cpus_allowed)) {
>
>                /* Copy values here to work around gcc-2.95.3, gcc-2.96 */
>                user = kstat_cpu(i).cpustat.user;
>
>  I'm sure that there are nicer ways of doing this, but Serge Hallyn
> suggested a while ago that I post here. Even though it says 2010, the
> patch above looks like it will go against 3.2.4 ok.
>
>  Thanks,
>     Andy
>
> --
> Andrew Phillips
> Head of Systems
>
> Direct: +44 (0)203 192 2509
> Mobile: +44 (0)7595 242 900
>
> LMAX, Yellow Building, 1A Nicholas Road,  London, W11 4AN
>
>
>
> The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company.
> _______________________________________________
> Containers mailing list
> Containers@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/containers

Hi,
We are also facing the same problem, the current JVM Ergonomics always
assume that it self is the only JVM running on the machine, which is
obviously wrong in a multiple LXC containers/hosting box. We are using
the options like -XX:UseParallelGC -XX:ParallelGCThreads=nnn etc to by
pass its adaptive algorithm.

--
Thanks,
Zhu Yanhai
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: cpu shielding.

[atp]

Serge,

> I'm afraid I haven't been following recent upstream discussions on
> this, but there are other people who want proc to show cgroup-limited
> information.  See for instance http://lwn.net/Articles/460310/ .  Glauber
> has also brought this up since then.  I'd recommend pinging him.
> 
> I'm all for /proc showing cgroup-filtered information, unconditionally.
> 

 I'll check with him, and report back. Having looked at how openvz
handles this, it seems that to do this comprehensively - even just for
cpu shielding is not a quick fix.

 Andy

-- 
Andrew Phillips
Head of Systems

Direct: +44 (0)203 192 2509
Mobile: +44 (0)7595 242 900

LMAX, Yellow Building, 1A Nicholas Road,  London, W11 4AN



The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company.

Re: cpu shielding.

[Glauber Costa]

On 02/07/2012 06:15 AM, Serge Hallyn wrote:
> Quoting atp (Andrew.Phillips-xheW4WVAX9Y@public.gmane.org):
>> Hello,
>>
>>    Apologies if I'm about to ask a frequently asked question - I did
>> check back over the last couple of months.
>>
>>    Is anyone working on cpu shielding for processes inside a cpu cgroup?
>>
>>    We would like to run Java in containers, and unfortunately it likes to
>> know how many processors there are in the system - to initialise thread
>> pools and such like.
>>
>>    I was thinking along these lines;
>>
>> --- fs/proc/stat.c.orig	2010-05-21 11:32:32.941258466 +0000
>> +++ fs/proc/stat.c	2010-05-21 11:40:47.681259133 +0000
>> @@ -39,7 +39,9 @@
>>   	getboottime(&boottime);
>>   	jif = boottime.tv_sec;
>>
>> -	for_each_possible_cpu(i) {
>> +//	for_each_possible_cpu(i) {
>> +//	// refer to the visible cpus.
>> +	for_each_cpu_and(i,cpu_possible_mask,(&current->cpus_allowed)) {
>>   		user = cputime64_add(user, kstat_cpu(i).cpustat.user);
>>   		nice = cputime64_add(nice, kstat_cpu(i).cpustat.nice);
>>   		system = cputime64_add(system, kstat_cpu(i).cpustat.system);
>> @@ -78,7 +80,10 @@
>>   		(unsigned long long)cputime64_to_clock_t(steal),
>>   		(unsigned long long)cputime64_to_clock_t(guest),
>>   		(unsigned long long)cputime64_to_clock_t(guest_nice));
>> -	for_each_online_cpu(i) {
>> +
>> +//	for_each_online_cpu(i) {
>> +//	// cgroup.
>> +	for_each_cpu_and(i,cpu_online_mask,(&current->cpus_allowed)) {
>>
>>   		/* Copy values here to work around gcc-2.95.3, gcc-2.96 */
>>   		user = kstat_cpu(i).cpustat.user;
>>
>>    I'm sure that there are nicer ways of doing this, but Serge Hallyn
>> suggested a while ago that I post here. Even though it says 2010, the
>> patch above looks like it will go against 3.2.4 ok.
>>
>>    Thanks,
>>       Andy
>
> I'm afraid I haven't been following recent upstream discussions on
> this, but there are other people who want proc to show cgroup-limited
> information.  See for instance http://lwn.net/Articles/460310/ .  Glauber
> has also brought this up since then.  I'd recommend pinging him.
>
> I'm all for /proc showing cgroup-filtered information, unconditionally.
>

Hi.

I have a patchset pending for review that deals with some part of that. 
(http://lwn.net/Articles/479624/)

The way I see it, there are two parts of the problem: One of them is 
keeping all those information consistently in the cgroup. I don't really 
like your patch, btw, because it takes the process as the main entity, 
and that is not really proc's idea. I'd go to the route of trying to 
devise a cpumask from the cgroup, and then expose this. That said, I 
believe anything in this area is far from a consensus.

Another problem is how to effectively display such data, after you 
gathered it. I am not essentially opposed to unconditionally displaying 
cgroup-filtered data as well, and I've sent a couple of patches trying 
to achieve that. But there are some problems with this approach that are 
preventing consensus now.