From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S932390Ab1FWCn7 (ORCPT <rfc822;w@1wt.eu>);
	Wed, 22 Jun 2011 22:43:59 -0400
Received: from mga14.intel.com ([143.182.124.37]:44394 "EHLO mga14.intel.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S932170Ab1FWCn6 (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Wed, 22 Jun 2011 22:43:58 -0400
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="4.65,410,1304319600"; 
   d="scan'208";a="17206994"
Subject: power increase issue on light load
From: "Alex,Shi" <alex.shi@intel.com>
To: ncrao@google.com, peterz@infradead.org, mingo@elte.hu
Cc: "Chen, Tim C" <tim.c.chen@intel.com>, "Li, Shaohua" <shaohua.li@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Date: Thu, 23 Jun 2011 10:43:44 +0800
Message-ID: <1308797024.23204.95.camel@debian>
Mime-Version: 1.0
X-Mailer: Evolution 2.28.1 
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

commit c8b281161dfa4bb5d5be63fb036ce19347b88c63 causes light load
benchmark use more than 10% system power on platform NHM-EP and laptop
Thinkpad T410 etc. The benchmarks are specpower and bltk office. 

I tried to track this issue, but only find deep C sate time reduced
much, about from 90% to 30~40%, the C0 or C1 state increase much on
different machines. 

Powertop just hints RES interrupts has a bit more. but when I try "perf
probe native_smp_send_reschedule". I didn't find much. 

I also checked the /proc/schedstat, just can sure the load_balance was
called a bit more frequency. but pull_task() was called really rare. 


The following are the /proc/schedstat increased number in about 300' when do bltk-office. 
The getting command is here:
#on a 16 LCPU system, with 3 level domain, 0,1,2, so all domain number
is 48, the domain statistic number is 2 + 36, so fs=38,

$cat /proc/schedstat > schedstat ; sleep x ; cat /proc/schedstat >>
schedstat ; cat schedstat | grep domain | sed '49 i \\n' |  awk -v fs=38
'BEGIN { RS=""; FS=" " } { if ( NR ==1) for (i=0; i<NF; i++)
{ value1[i]=$i ; } ;  if ( NR ==2) for (i=0; i<NF; i++) { value2[i]=
$i } } END {ORS=" ";  for (i=0;i<NF;i++){ if (i%fs == 0)  ll="\n"; else
ll=""; print value2[i] - value1[i]  ll  };  print "\n" }'

BTW, the imbalance increasing is due to the SCALE increase about 1024. 

=============== schedsat on 2.6.39 ========= 
0 0 7819 7812 6 7168 1 0 0 7812 617 614 3 3072 0 0 0 614 19617 19165 103 462848 349 1 0 19165 0 0 0 0 0 0 0 0 0 15484 819 0
 0 0 6262 6243 13 20716 6 0 0 6243 80 80 0 0 0 0 0 17 19268 19210 53 62326 5 0 0 19210 0 0 0 0 0 0 0 0 0 4898 10 0
 0 0 1418 1416 1 2230 1 0 0 1416 1 1 0 0 0 0 0 0 19263 19262 1 1531 0 0 1 19261 0 0 0 0 0 0 0 0 0 878 439 0
 0 0 1351 1348 3 3072 1 1 0 1348 13 13 0 0 0 0 0 13 2160 2159 1 1024 0 0 0 2159 1 0 1 0 0 0 0 0 0 1265 11 0
 0 0 1066 1066 0 511 0 0 0 1066 9 9 0 0 0 0 0 4 2160 2159 1 1024 0 0 0 2159 0 0 0 0 0 0 0 0 0 393 1 0
 0 0 532 506 15 24850 11 0 0 506 2 2 0 0 0 0 0 2 2160 2157 2 4085 1 0 0 2157 0 0 0 0 0 0 0 0 0 942 435 0
 0 0 1963 1962 1 1024 0 0 0 1962 267 267 0 0 0 0 0 267 8372 8367 5 5120 0 0 0 8367 0 0 0 0 0 0 0 0 0 1502 2 0
 0 0 1865 1712 143 167805 14 5 0 1712 185 185 0 0 0 0 0 6 8372 8259 109 120287 4 0 0 8259 4 0 4 0 0 0 0 0 0 5855 0 0
 0 0 476 476 0 501 0 0 0 122 10 10 0 0 0 0 0 1 8368 8368 0 0 0 0 0 8368 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1227 1227 0 0 0 0 0 1227 1 1 0 0 0 0 0 1 301 301 0 0 0 0 0 301 0 0 0 0 0 0 0 0 0 7 0 0
 0 0 1136 1124 11 14075 1 0 0 1124 1 1 0 0 0 0 0 0 301 298 3 3072 0 0 0 298 0 0 0 0 0 0 0 0 0 396 0 0
 0 0 489 489 0 0 0 0 0 8 0 0 0 0 0 0 0 0 301 301 0 0 0 0 0 301 0 0 0 0 0 0 0 0 0 36 0 0
 0 0 1374 1374 0 0 0 0 0 1374 5 5 0 0 0 0 0 5 428 428 0 0 0 0 0 428 0 0 0 0 0 0 0 0 0 34 0 0
 0 0 1301 1151 134 163543 17 4 1 1150 3 3 0 0 0 0 0 0 428 352 72 78311 4 0 0 352 1 0 1 0 0 0 0 0 0 961 0 0
 0 0 407 407 0 0 0 0 0 9 0 0 0 0 0 0 0 0 424 424 0 0 0 0 0 424 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 1100 1100 0 0 0 0 0 1100 0 0 0 0 0 0 0 0 159 159 0 0 0 0 0 159 0 0 0 0 0 0 0 0 0 2 0 0
 0 0 1092 1071 20 24826 1 0 0 1071 0 0 0 0 0 0 0 0 159 157 2 1914 0 0 0 157 0 0 0 0 0 0 0 0 0 26 0 0
 0 0 470 470 0 0 0 0 0 0 0 0 0 0 0 0 0 0 159 158 1 2047 0 0 1 157 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 1257 1257 0 0 0 0 0 1257 0 0 0 0 0 0 0 0 196 196 0 0 0 0 0 196 0 0 0 0 0 0 0 0 0 23 0 0
 0 0 1210 1051 151 175150 12 5 1 1050 0 0 0 0 0 0 0 0 196 159 35 37885 2 0 0 159 4 0 4 0 0 0 0 0 0 55 0 0
 0 0 486 486 0 0 0 0 0 1 0 0 0 0 0 0 0 0 194 194 0 0 0 0 0 194 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 988 988 0 0 0 0 0 988 0 0 0 0 0 0 0 0 115 115 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 972 949 23 26105 0 0 0 949 0 0 0 0 0 0 0 0 115 115 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 457 457 0 0 0 0 0 0 0 0 0 0 0 0 0 0 115 115 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 4431 4411 10 20992 10 1 0 4411 522 520 0 2560 2 0 0 520 17075 16946 117 133119 12 0 3 16943 0 0 0 0 0 0 0 0 0 15708 2 0
 0 0 4421 4415 6 7424 0 0 0 697 353 353 0 0 0 0 0 5 17063 17050 12 15225 1 0 1 17049 0 0 0 0 0 0 0 0 0 24 0 0
 0 0 283 283 0 0 0 0 0 0 42 42 0 0 0 0 0 0 17062 17062 0 0 0 0 0 17062 0 0 0 0 0 0 0 0 0 65 0 0
 0 0 1272 1271 1 1024 0 0 0 1271 7 7 0 0 0 0 0 7 534 534 0 0 0 0 0 534 0 0 0 0 0 0 0 0 0 1175 0 0
 0 0 1223 1223 0 0 0 0 0 13 5 5 0 0 0 0 0 0 534 533 1 1535 0 0 0 533 0 0 0 0 0 0 0 0 0 15 0 0
 0 0 31 31 0 0 0 0 0 0 5 5 0 0 0 0 0 0 534 534 0 0 0 0 0 534 0 0 0 0 0 0 0 0 0 23 0 0
 0 0 1743 1742 1 1024 0 0 0 1742 106 105 0 1024 1 0 0 105 2575 2570 4 5120 1 0 0 2570 0 0 0 0 0 0 0 0 0 1369 0 0
 0 0 1743 1742 1 1024 0 0 0 151 67 67 0 0 0 0 0 0 2574 2554 17 20344 3 0 0 2554 0 0 0 0 0 0 0 0 0 184 0 0
 0 0 49 49 0 0 0 0 0 0 38 38 0 0 0 0 0 0 2571 2570 0 1536 1 0 0 2570 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1126 1126 0 0 0 0 0 1126 2 2 0 0 0 0 0 2 211 210 0 1024 1 0 0 210 0 0 0 0 0 0 0 0 0 42 0 0
 0 0 1126 1126 0 0 0 0 0 1 2 2 0 0 0 0 0 0 210 210 0 0 0 0 0 210 0 0 0 0 0 0 0 0 0 2 0 0
 0 0 1 1 0 0 0 0 0 0 2 2 0 0 0 0 0 0 210 210 0 0 0 0 0 210 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1177 1175 2 2048 0 0 0 1175 0 0 0 0 0 0 0 0 222 222 0 0 0 0 0 222 0 0 0 0 0 0 0 0 0 27 0 0
 0 0 1177 1177 0 0 0 0 0 17 0 0 0 0 0 0 0 0 222 209 9 12916 4 0 1 208 0 0 0 0 0 0 0 0 0 18 0 0
 0 0 11 11 0 0 0 0 0 0 1 1 0 0 0 0 0 0 218 218 0 0 0 0 0 218 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1070 1069 1 1024 0 0 0 1069 0 0 0 0 0 0 0 0 123 123 0 0 0 0 0 123 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1070 1070 0 0 0 0 0 2 0 0 0 0 0 0 0 0 123 122 1 1024 0 0 0 122 0 0 0 0 0 0 0 0 0 3 0 0
 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 123 123 0 0 0 0 0 123 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1059 1059 0 0 0 0 0 1059 0 0 0 0 0 0 0 0 147 147 0 0 0 0 0 147 0 0 0 0 0 0 0 0 0 22 0 0
 0 0 1059 1059 0 0 0 0 0 4 0 0 0 0 0 0 0 0 147 141 6 6010 0 0 0 141 0 0 0 0 0 0 0 0 0 7 0 0
 0 0 5 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 147 147 0 0 0 0 0 147 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 987 987 0 0 0 0 0 987 0 0 0 0 0 0 0 0 115 115 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 987 987 0 0 0 0 0 0 0 0 0 0 0 0 0 0 115 115 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 115 115 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0
                                  
============= for 3.0.0-rc4 ===================


 0 0 9928 9926 2 2097152 0 0 0 9926 617 617 0 0 0 0 0 617 15813 15730 75 87031808 8 0 0 15730 0 0 0 0 0 0 0 0 0 15578 395 0
 0 0 9882 9870 11 14942201 1 0 0 9870 489 489 0 0 0 0 0 18 15805 15794 10 11397255 1 0 0 15794 0 0 0 0 0 0 0 0 0 5812 22 0
 0 0 1500 1498 1 2473919 1 0 0 1498 1 1 0 0 0 0 0 0 15804 15799 2 7864320 3 0 0 15799 0 0 0 0 0 0 0 0 0 1388 66 0
 0 0 1764 1763 1 1048576 0 0 0 1763 102 102 0 0 0 0 0 102 4932 4930 1 2097152 1 0 0 4930 0 0 0 0 0 0 0 0 0 996 6 0
 0 0 1764 1749 8 15728632 7 0 0 1749 90 90 0 0 0 0 0 6 4931 4877 54 56885246 0 0 0 4877 0 0 0 0 0 0 0 0 0 2183 11 0
 0 0 652 631 15 21226946 6 0 0 631 5 5 0 0 0 0 0 0 4931 4764 167 181927108 0 0 1 4763 0 0 0 0 0 0 0 0 0 427 60 0
 0 0 3577 3575 2 2097152 0 0 0 3575 113 113 0 0 0 0 0 113 2396 2390 6 6291456 0 0 1 2389 0 0 0 0 0 0 0 0 0 243 0 0
 0 0 2896 2753 134 157411412 10 1 3 2750 96 96 0 0 0 0 0 1 2396 2338 56 62914553 2 0 0 2338 1 0 1 0 0 0 0 0 0 6115 0 0
 0 0 644 644 0 0 0 0 2 174 2 2 0 0 0 0 0 1 2394 2389 2 8912888 3 0 0 2389 0 0 0 0 0 0 0 0 0 9 0 0
 0 0 1573 1573 0 0 0 0 0 1573 170 170 0 0 0 0 0 170 4734 4731 1 3145728 2 0 0 4731 0 0 0 0 0 0 0 0 0 437 0 0
 0 0 1451 1440 9 11796478 2 0 0 1440 96 96 0 0 0 0 0 1 4732 4699 33 35913726 0 0 0 4699 0 0 0 0 0 0 0 0 0 3137 0 0
 0 0 558 558 0 0 0 0 0 13 17 17 0 0 0 0 0 0 4732 4729 2 4194291 1 0 0 4729 0 0 0 0 0 0 0 0 0 846 0 0
 0 0 1690 1690 0 0 0 0 0 1690 1 1 0 0 0 0 0 1 297 297 0 0 0 0 0 297 0 0 0 0 0 0 0 0 0 56 0 0
 0 0 1593 1440 144 171092595 13 5 0 1440 1 1 0 0 0 0 0 0 297 274 21 23330810 2 0 0 274 4 0 4 0 0 0 0 0 0 510 0 0
 0 0 580 580 0 0 0 0 0 5 1 1 0 0 0 0 0 0 295 294 1 1048571 0 0 0 294 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1660 1660 0 0 0 0 0 1660 1 1 0 0 0 0 0 1 205 205 0 0 0 0 0 205 0 0 0 0 0 0 0 0 0 4 0 0
 0 0 1660 1629 26 32243706 5 0 0 1629 1 1 0 0 0 0 0 0 205 185 17 20310149 3 0 0 185 0 0 0 0 0 0 0 0 0 185 0 0
 0 0 573 573 0 0 0 0 0 3 1 1 0 0 0 0 0 0 202 202 0 0 0 0 0 202 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 1483 1483 0 0 0 0 0 1483 0 0 0 0 0 0 0 0 146 146 0 0 0 0 0 146 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 1375 1223 147 173014981 7 3 1 1222 0 0 0 0 0 0 0 0 146 134 8 12183684 4 0 0 134 2 0 2 0 0 0 0 0 0 21 0 0
 0 0 558 558 0 0 0 0 0 1 0 0 0 0 0 0 0 0 142 142 0 0 0 0 0 142 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1372 1372 0 0 0 0 0 1372 0 0 0 0 0 0 0 0 129 129 0 0 0 0 0 129 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1372 1343 22 30408700 8 3 0 1343 0 0 0 0 0 0 0 0 129 124 3 5242879 2 0 0 124 1 0 1 0 0 0 0 0 0 11 0 0
 0 0 542 542 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 127 0 0 0 0 0 127 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 12209 12190 16 20971520 3 0 0 12190 296 294 1 2097152 1 0 0 294 16022 15892 123 136839168 7 0 12 15880 0 0 0 0 0 0 0 0 0 16130 2 0
 0 0 12206 12190 13 20185088 3 0 0 1180 195 195 0 0 0 0 0 0 16015 16006 8 11010047 1 0 1 16005 0 0 0 0 0 0 0 0 0 34 0 0
 0 0 678 678 0 0 0 0 0 0 16 16 0 0 0 0 0 0 16014 16012 1 3145718 1 0 0 16012 0 0 0 0 0 0 0 0 0 32 1 0
 0 0 1830 1828 2 2097152 0 0 0 1828 123 123 0 0 0 0 0 123 4970 4967 2 3145728 1 0 0 4967 0 0 0 0 0 0 0 0 0 1157 0 0
 0 0 1830 1829 0 911495 1 0 0 65 95 95 0 0 0 0 0 0 4969 4966 2 2883582 1 0 0 4966 0 0 0 0 0 0 0 0 0 9 0 0
 0 0 24 24 0 0 0 0 0 0 38 38 0 0 0 0 0 0 4968 4965 2 3670011 1 0 0 4965 0 0 0 0 0 0 0 0 0 2 0 0
 0 0 2816 2813 3 3145728 0 0 0 2813 4 4 0 0 0 0 0 4 544 544 0 0 0 0 0 544 0 0 0 0 0 0 0 0 0 224 0 0
 0 0 2816 2814 1 1960071 1 0 0 78 3 3 0 0 0 0 0 0 544 534 9 10747903 1 0 0 534 0 0 0 0 0 0 0 0 0 31 0 0
 0 0 30 30 0 0 0 0 0 0 5 5 0 0 0 0 0 0 543 542 1 1048576 0 0 0 542 0 0 0 0 0 0 0 0 0 8 0 0
 0 0 1551 1550 1 1048576 0 0 0 1550 8 8 0 0 0 0 0 8 672 669 2 3145728 1 0 0 669 0 0 0 0 0 0 0 0 0 411 0 0
 0 0 1551 1551 0 0 0 0 0 18 7 7 0 0 0 0 0 0 671 668 3 3145727 0 0 0 668 0 0 0 0 0 0 0 0 0 302 0 0
 0 0 14 14 0 0 0 0 0 0 3 3 0 0 0 0 0 0 671 671 0 524288 0 0 0 671 0 0 0 0 0 0 0 0 0 3 0 0
 0 0 1378 1378 0 0 0 0 0 1378 0 0 0 0 0 0 0 0 169 169 0 0 0 0 0 169 0 0 0 0 0 0 0 0 0 58 0 0
 0 0 1357 1356 0 1048576 1 0 0 6 0 0 0 0 0 0 0 0 169 163 6 7864317 0 0 0 163 0 0 0 0 0 0 0 0 0 18 0 0
 0 0 15 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 169 169 0 0 0 0 0 169 0 0 0 0 0 0 0 0 0 3 0 0
 0 0 1232 1232 0 0 0 0 0 1232 0 0 0 0 0 0 0 0 126 126 0 0 0 0 0 126 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 1232 1232 0 0 0 0 0 2 0 0 0 0 0 0 0 0 126 123 3 2883583 0 0 0 123 0 0 0 0 0 0 0 0 0 3 0 0
 0 0 2 2 0 0 0 0 0 0 1 1 0 0 0 0 0 0 126 126 0 0 0 0 0 126 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 1222 1222 0 0 0 0 0 1222 1 1 0 0 0 0 0 1 122 122 0 0 0 0 0 122 0 0 0 0 0 0 0 0 0 1 0 0
 0 0 1113 1112 0 911495 1 0 0 13 1 1 0 0 0 0 0 0 122 118 4 4319367 0 0 0 118 0 0 0 0 0 0 0 0 0 2 0 0
 0 0 47 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 122 122 0 0 0 0 0 122 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1222 1222 0 0 0 0 0 1222 0 0 0 0 0 0 0 0 118 118 0 0 0 0 0 118 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 1222 1222 0 0 0 0 0 0 0 0 0 0 0 0 0 0 118 116 2 2097152 0 0 0 116 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 118 118 0 0 0 0 0 118 0 0 0 0 0 0 0 0 0 0 0



Any ideas of this? 

Regards
Alex