public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* Re: [PATCH v3 1/3] cpufreq: ondemand: Change the calculation of target frequency
@ 2013-06-08 12:34 Stratos Karafotis
  2013-06-08 14:05 ` Rafael J. Wysocki
  0 siblings, 1 reply; 43+ messages in thread
From: Stratos Karafotis @ 2013-06-08 12:34 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Borislav Petkov, Viresh Kumar, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linux-pm, cpufreq, linux-kernel

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 5698 bytes --]

I also did the test with the way you mentioned. But I thought to run turbostat for 100 sec as I did with powertop.
Actually benchmark lasts about 96 secs.

I think that we use almost the same energy for 100 sec to run the same load a little bit faster. I think this means also a reduce to power consumption.

I will also send the results running the test as you said.

Thanks again,
Stratos

"Rafael J. Wysocki" <rjw@sisk.pl> wrote:

>On Saturday, June 08, 2013 12:56:00 PM Stratos Karafotis wrote:
>> On 06/07/2013 11:57 PM, Rafael J. Wysocki wrote:
>> > On Friday, June 07, 2013 10:14:34 PM Stratos Karafotis wrote:
>> >> On 06/05/2013 11:35 PM, Rafael J. Wysocki wrote:
>> >>> On Wednesday, June 05, 2013 08:13:26 PM Stratos Karafotis wrote:
>> >>>> Hi Borislav,
>> >>>>
>> >>>> On 06/05/2013 07:17 PM, Borislav Petkov wrote:
>> >>>>> On Wed, Jun 05, 2013 at 07:01:25PM +0300, Stratos Karafotis wrote:
>> >>>>>> Ondemand calculates load in terms of frequency and increases it only
>> >>>>>> if the load_freq is greater than up_threshold multiplied by current
>> >>>>>> or average frequency. This seems to produce oscillations of frequency
>> >>>>>> between min and max because, for example, a relatively small load can
>> >>>>>> easily saturate minimum frequency and lead the CPU to max. Then, the
>> >>>>>> CPU will decrease back to min due to a small load_freq.
>> >>>>>
>> >>>>> Right, and I think this is how we want it, no?
>> >>>>>
>> >>>>> The thing is, the faster you finish your work, the faster you can become
>> >>>>> idle and save power.
>> >>>>
>> >>>> This is exactly the goal of this patch. To use more efficiently middle
>> >>>> frequencies to finish faster the work.
>> >>>>
>> >>>>> If you switch frequencies in a staircase-like manner, you're going to
>> >>>>> take longer to finish, in certain cases, and burn more power while doing
>> >>>>> so.
>> >>>>
>> >>>> This is not true with this patch. It switches to middle frequencies
>> >>>> when the load < up_threshold.
>> >>>> Now, ondemand does not increase freq. CPU runs in lowest freq till the
>> >>>> load is greater than up_threshold.
>> >>>>
>> >>>>> Btw, racing to idle is also a good example for why you want boosting:
>> >>>>> you want to go max out the core but stay within power limits so that you
>> >>>>> can finish sooner.
>> >>>>>
>> >>>>>> This patch changes the calculation method of load and target frequency
>> >>>>>> considering 2 points:
>> >>>>>> - Load computation should be independent from current or average
>> >>>>>> measured frequency. For example an absolute load 80% at 100MHz is not
>> >>>>>> necessarily equivalent to 8% at 1000MHz in the next sampling interval.
>> >>>>>> - Target frequency should be increased to any value of frequency table
>> >>>>>> proportional to absolute load, instead to only the max. Thus:
>> >>>>>>
>> >>>>>> Target frequency = C * load
>> >>>>>>
>> >>>>>> where C = policy->cpuinfo.max_freq / 100
>> >>>>>>
>> >>>>>> Tested on Intel i7-3770 CPU @ 3.40GHz and on Quad core 1500MHz Krait.
>> >>>>>> Phoronix benchmark of Linux Kernel Compilation 3.1 test shows an
>> >>>>>> increase ~1.5% in performance. cpufreq_stats (time_in_state) shows
>> >>>>>> that middle frequencies are used more, with this patch. Highest
>> >>>>>> and lowest frequencies were used less by ~9%
>> >>>
>> >>> Can you also use powertop to measure the percentage of time spent in idle
>> >>> states for the same workload with and without your patchset?  Also, it would
>> >>> be good to measure the total energy consumption somehow ...
>> >>>
>> >>> Thanks,
>> >>> Rafael
>> >>
>> >> Hi Rafael,
>> >>
>> >> I repeated the tests extracting also powertop results.
>> >> Measurement steps with and without this patch:
>> >> 1) Reboot system
>> >> 2) Running twice Phoronix benchmark of Linux Kernel Compilation 3.1 test
>> >>     without taking measurement
>> >> 3) Wait few minutes
>> >> 4) Run Phoronix and powertop for 100secs and take measurement.
>> > 
>> > Well, while this is not conclusive, it definitely looks very promising. :-)
>> > 
>> > We're seeing measurable performance improvement with the patchset applied *and*
>> > more time spent in idle states both at the same time.  I'd be very surprised if
>> > the energy consumption measuremets did not confirm that the patchset allowed
>> > us to reduce it.
>> > 
>> > If my computations are correct (somebody please check), the cores spent about
>> > 20% more time in idle on the average with the patchset applied and in addition
>> > to that the cc6 residency was greater by about 2% on the average with respect
>> > to the kernel without the patchset.
>> > 
>> > We need to verify if there are gains (or at least no regressions) with other
>> > workloads, but since this *also* reduces code complexity quite a bit, I'm
>> > seriously considering taking it.
>> > 
>> >> I will try to repeat the test and take measurements with turbostat as
>> >> Borislav suggested.
>> > 
>> > Please do!
>> > 
>> > Thanks,
>> > Rafael
>> > 
>> 
>> Hi,
>> 
>> I repeated the tests extracting results from turbostat.
>> Measurement steps with and without this patch:
>> 1) Reboot system
>> 2) Running twice Phoronix benchmark of Linux Kernel Compilation 3.1 test
>>    without taking measurement
>> 3) Wait few minutes
>> 4) Run Phoronix and turbostat (-i 100) and take measurement
>
>You need to do something like
>
># ./turbostat <command invoking the phoronix suite>
>
>Did you do that?
>
>Rafael
>
>
>-- 
>I speak only for myself.
>Rafael J. Wysocki, Intel Open Source Technology Center.
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 43+ messages in thread
* Re: [PATCH v3 1/3] cpufreq: ondemand: Change the calculation of target frequency
@ 2013-06-06 12:56 Stratos Karafotis
  0 siblings, 0 replies; 43+ messages in thread
From: Stratos Karafotis @ 2013-06-06 12:56 UTC (permalink / raw)
  To: Viresh Kumar
  Cc: Borislav Petkov, Rafael J. Wysocki, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linux-pm, cpufreq, linux-kernel

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 1436 bytes --]

Thanks Viresh. I think I couldn't explain this in better way.
Also thanks for acknowledgment!

Stratos

Viresh Kumar <viresh.kumar@linaro.org> wrote:

>On 6 June 2013 15:31, Borislav Petkov <bp@suse.de> wrote:
>
>> Hold on, you say above "easily saturate minimum frequency and lead the
>> CPU to max". I read this as we jump straight to max P-state where we
>> even boost.
>
>Probably he meant: "At lowest levels of frequencies, a small load on system
>may look like a huge one. like: 20-30% load on max freq can be 95% load
>on min freq. And so we jump to max freq even for this load and return back
>pretty quickly as this load doesn't sustain for longer. over that we wait for
>load to go over up_threshold to increase freq."
>
>> "CPU to max" finishes the work faster than middle frequencies, if you're
>> CPU-bound.
>
>He isn't removing this feature at all.
>
>Current code is:
>
>if (load > up_threshold)
>   goto maxfreq.
>else
>   don't increase freq, maybe decrease it in steps
>
>What he is doing is:
>
>if (load > up_threshold)
>   goto maxfreq.
>else
>   increase/decrease freq based on current load.
>
>So, if up_threshold is 95 and load remains < 95, his patch will
>give significant improvement both power & performance wise.
>
>Else, it shouldn't decrease it.
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 43+ messages in thread
* Re: [PATCH v3 1/3] cpufreq: ondemand: Change the calculation of target frequency
@ 2013-06-06 12:54 Stratos Karafotis
  2013-06-06 13:15 ` Borislav Petkov
  0 siblings, 1 reply; 43+ messages in thread
From: Stratos Karafotis @ 2013-06-06 12:54 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Borislav Petkov, Viresh Kumar, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linux-pm, cpufreq, linux-kernel

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 3075 bytes --]

Hi Rafael,

I will try to provide the requested info (although, I'm not sure how to measure total energy :) )

Thanks,
Stratos

"Rafael J. Wysocki" <rjw@sisk.pl> wrote:

>On Wednesday, June 05, 2013 08:13:26 PM Stratos Karafotis wrote:
>> Hi Borislav,
>> 
>> On 06/05/2013 07:17 PM, Borislav Petkov wrote:
>> > On Wed, Jun 05, 2013 at 07:01:25PM +0300, Stratos Karafotis wrote:
>> >> Ondemand calculates load in terms of frequency and increases it only
>> >> if the load_freq is greater than up_threshold multiplied by current
>> >> or average frequency. This seems to produce oscillations of frequency
>> >> between min and max because, for example, a relatively small load can
>> >> easily saturate minimum frequency and lead the CPU to max. Then, the
>> >> CPU will decrease back to min due to a small load_freq.
>> >
>> > Right, and I think this is how we want it, no?
>> >
>> > The thing is, the faster you finish your work, the faster you can become
>> > idle and save power.
>> 
>> This is exactly the goal of this patch. To use more efficiently middle
>> frequencies to finish faster the work.
>> 
>> > If you switch frequencies in a staircase-like manner, you're going to
>> > take longer to finish, in certain cases, and burn more power while doing
>> > so.
>> 
>> This is not true with this patch. It switches to middle frequencies
>> when the load < up_threshold.
>> Now, ondemand does not increase freq. CPU runs in lowest freq till the
>> load is greater than up_threshold.
>> 
>> > Btw, racing to idle is also a good example for why you want boosting:
>> > you want to go max out the core but stay within power limits so that you
>> > can finish sooner.
>> >
>> >> This patch changes the calculation method of load and target frequency
>> >> considering 2 points:
>> >> - Load computation should be independent from current or average
>> >> measured frequency. For example an absolute load 80% at 100MHz is not
>> >> necessarily equivalent to 8% at 1000MHz in the next sampling interval.
>> >> - Target frequency should be increased to any value of frequency table
>> >> proportional to absolute load, instead to only the max. Thus:
>> >>
>> >> Target frequency = C * load
>> >>
>> >> where C = policy->cpuinfo.max_freq / 100
>> >>
>> >> Tested on Intel i7-3770 CPU @ 3.40GHz and on Quad core 1500MHz Krait.
>> >> Phoronix benchmark of Linux Kernel Compilation 3.1 test shows an
>> >> increase ~1.5% in performance. cpufreq_stats (time_in_state) shows
>> >> that middle frequencies are used more, with this patch. Highest
>> >> and lowest frequencies were used less by ~9%
>
>Can you also use powertop to measure the percentage of time spent in idle
>states for the same workload with and without your patchset?  Also, it would
>be good to measure the total energy consumption somehow ...
>
>Thanks,
>Rafael
>
>
>-- 
>I speak only for myself.
>Rafael J. Wysocki, Intel Open Source Technology Center.
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

^ permalink raw reply	[flat|nested] 43+ messages in thread
* [PATCH v3 1/3] cpufreq: ondemand: Change the calculation of target frequency
@ 2013-06-05 16:01 Stratos Karafotis
  2013-06-05 16:17 ` Borislav Petkov
  0 siblings, 1 reply; 43+ messages in thread
From: Stratos Karafotis @ 2013-06-05 16:01 UTC (permalink / raw)
  To: Rafael J. Wysocki, Viresh Kumar
  Cc: Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Borislav Petkov,
	linux-pm, cpufreq, linux-kernel

Ondemand calculates load in terms of frequency and increases it only
if the load_freq is greater than up_threshold multiplied by current
or average frequency. This seems to produce oscillations of frequency
between min and max because, for example, a relatively small load can
easily saturate minimum frequency and lead the CPU to max. Then, the
CPU will decrease back to min due to a small load_freq.

This patch changes the calculation method of load and target frequency
considering 2 points:
- Load computation should be independent from current or average
measured frequency. For example an absolute load 80% at 100MHz is not
necessarily equivalent to 8% at 1000MHz in the next sampling interval.
- Target frequency should be increased to any value of frequency table
proportional to absolute load, instead to only the max. Thus:

Target frequency = C * load

where C = policy->cpuinfo.max_freq / 100

Tested on Intel i7-3770 CPU @ 3.40GHz and on Quad core 1500MHz Krait.
Phoronix benchmark of Linux Kernel Compilation 3.1 test shows an
increase ~1.5% in performance. cpufreq_stats (time_in_state) shows
that middle frequencies are used more, with this patch. Highest
and lowest frequencies were used less by ~9%

Signed-off-by: Stratos Karafotis <stratosk@semaphore.gr>
---
 drivers/cpufreq/cpufreq_governor.c | 10 +---------
 drivers/cpufreq/cpufreq_governor.h |  1 -
 drivers/cpufreq/cpufreq_ondemand.c | 39 +++++++-------------------------------
 3 files changed, 8 insertions(+), 42 deletions(-)

diff --git a/drivers/cpufreq/cpufreq_governor.c b/drivers/cpufreq/cpufreq_governor.c
index a849b2d..47c8077 100644
--- a/drivers/cpufreq/cpufreq_governor.c
+++ b/drivers/cpufreq/cpufreq_governor.c
@@ -54,7 +54,7 @@ void dbs_check_cpu(struct dbs_data *dbs_data, int cpu)
 
 	policy = cdbs->cur_policy;
 
-	/* Get Absolute Load (in terms of freq for ondemand gov) */
+	/* Get Absolute Load */
 	for_each_cpu(j, policy->cpus) {
 		struct cpu_dbs_common_info *j_cdbs;
 		u64 cur_wall_time, cur_idle_time;
@@ -105,14 +105,6 @@ void dbs_check_cpu(struct dbs_data *dbs_data, int cpu)
 
 		load = 100 * (wall_time - idle_time) / wall_time;
 
-		if (dbs_data->cdata->governor == GOV_ONDEMAND) {
-			int freq_avg = __cpufreq_driver_getavg(policy, j);
-			if (freq_avg <= 0)
-				freq_avg = policy->cur;
-
-			load *= freq_avg;
-		}
-
 		if (load > max_load)
 			max_load = load;
 	}
diff --git a/drivers/cpufreq/cpufreq_governor.h b/drivers/cpufreq/cpufreq_governor.h
index e7bbf76..c305cad 100644
--- a/drivers/cpufreq/cpufreq_governor.h
+++ b/drivers/cpufreq/cpufreq_governor.h
@@ -169,7 +169,6 @@ struct od_dbs_tuners {
 	unsigned int sampling_rate;
 	unsigned int sampling_down_factor;
 	unsigned int up_threshold;
-	unsigned int adj_up_threshold;
 	unsigned int powersave_bias;
 	unsigned int io_is_busy;
 };
diff --git a/drivers/cpufreq/cpufreq_ondemand.c b/drivers/cpufreq/cpufreq_ondemand.c
index 4b9bb5d..62e67a9 100644
--- a/drivers/cpufreq/cpufreq_ondemand.c
+++ b/drivers/cpufreq/cpufreq_ondemand.c
@@ -29,11 +29,9 @@
 #include "cpufreq_governor.h"
 
 /* On-demand governor macros */
-#define DEF_FREQUENCY_DOWN_DIFFERENTIAL		(10)
 #define DEF_FREQUENCY_UP_THRESHOLD		(80)
 #define DEF_SAMPLING_DOWN_FACTOR		(1)
 #define MAX_SAMPLING_DOWN_FACTOR		(100000)
-#define MICRO_FREQUENCY_DOWN_DIFFERENTIAL	(3)
 #define MICRO_FREQUENCY_UP_THRESHOLD		(95)
 #define MICRO_FREQUENCY_MIN_SAMPLE_RATE		(10000)
 #define MIN_FREQUENCY_UP_THRESHOLD		(11)
@@ -159,14 +157,10 @@ static void dbs_freq_increase(struct cpufreq_policy *p, unsigned int freq)
 
 /*
  * Every sampling_rate, we check, if current idle time is less than 20%
- * (default), then we try to increase frequency. Every sampling_rate, we look
- * for the lowest frequency which can sustain the load while keeping idle time
- * over 30%. If such a frequency exist, we try to decrease to this frequency.
- *
- * Any frequency increase takes it to the maximum frequency. Frequency reduction
- * happens at minimum steps of 5% (default) of current frequency
+ * (default), then we try to increase frequency. Else, we adjust the frequency
+ * proportional to load.
  */
-static void od_check_cpu(int cpu, unsigned int load_freq)
+static void od_check_cpu(int cpu, unsigned int load)
 {
 	struct od_cpu_dbs_info_s *dbs_info = &per_cpu(od_cpu_dbs_info, cpu);
 	struct cpufreq_policy *policy = dbs_info->cdbs.cur_policy;
@@ -176,29 +170,17 @@ static void od_check_cpu(int cpu, unsigned int load_freq)
 	dbs_info->freq_lo = 0;
 
 	/* Check for frequency increase */
-	if (load_freq > od_tuners->up_threshold * policy->cur) {
+	if (load > od_tuners->up_threshold) {
 		/* If switching to max speed, apply sampling_down_factor */
 		if (policy->cur < policy->max)
 			dbs_info->rate_mult =
 				od_tuners->sampling_down_factor;
 		dbs_freq_increase(policy, policy->max);
 		return;
-	}
-
-	/* Check for frequency decrease */
-	/* if we cannot reduce the frequency anymore, break out early */
-	if (policy->cur == policy->min)
-		return;
-
-	/*
-	 * The optimal frequency is the frequency that is the lowest that can
-	 * support the current CPU usage without triggering the up policy. To be
-	 * safe, we focus 10 points under the threshold.
-	 */
-	if (load_freq < od_tuners->adj_up_threshold
-			* policy->cur) {
+	} else {
+		/* Calculate the next frequency proportional to load */
 		unsigned int freq_next;
-		freq_next = load_freq / od_tuners->adj_up_threshold;
+		freq_next = load * policy->cpuinfo.max_freq / 100;
 
 		/* No longer fully busy, reset rate_mult */
 		dbs_info->rate_mult = 1;
@@ -372,9 +354,6 @@ static ssize_t store_up_threshold(struct dbs_data *dbs_data, const char *buf,
 			input < MIN_FREQUENCY_UP_THRESHOLD) {
 		return -EINVAL;
 	}
-	/* Calculate the new adj_up_threshold */
-	od_tuners->adj_up_threshold += input;
-	od_tuners->adj_up_threshold -= od_tuners->up_threshold;
 
 	od_tuners->up_threshold = input;
 	return count;
@@ -523,8 +502,6 @@ static int od_init(struct dbs_data *dbs_data)
 	if (idle_time != -1ULL) {
 		/* Idle micro accounting is supported. Use finer thresholds */
 		tuners->up_threshold = MICRO_FREQUENCY_UP_THRESHOLD;
-		tuners->adj_up_threshold = MICRO_FREQUENCY_UP_THRESHOLD -
-			MICRO_FREQUENCY_DOWN_DIFFERENTIAL;
 		/*
 		 * In nohz/micro accounting case we set the minimum frequency
 		 * not depending on HZ, but fixed (very low). The deferred
@@ -533,8 +510,6 @@ static int od_init(struct dbs_data *dbs_data)
 		dbs_data->min_sampling_rate = MICRO_FREQUENCY_MIN_SAMPLE_RATE;
 	} else {
 		tuners->up_threshold = DEF_FREQUENCY_UP_THRESHOLD;
-		tuners->adj_up_threshold = DEF_FREQUENCY_UP_THRESHOLD -
-			DEF_FREQUENCY_DOWN_DIFFERENTIAL;
 
 		/* For correct statistics, we need 10 ticks for each measure */
 		dbs_data->min_sampling_rate = MIN_SAMPLING_RATE_RATIO *
-- 
1.8.1.4


^ permalink raw reply related	[flat|nested] 43+ messages in thread
end of thread, other threads:[~2015-02-23 16:45 UTC | newest]

Thread overview: 43+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-06-08 12:34 [PATCH v3 1/3] cpufreq: ondemand: Change the calculation of target frequency Stratos Karafotis
2013-06-08 14:05 ` Rafael J. Wysocki
2013-06-08 20:31   ` Stratos Karafotis
2013-06-08 22:18     ` Rafael J. Wysocki
2013-06-09 16:26       ` Borislav Petkov
2013-06-09 18:08         ` Stratos Karafotis
2013-06-09 20:58           ` Rafael J. Wysocki
2013-06-09 21:14             ` Borislav Petkov
2013-06-09 22:11               ` Rafael J. Wysocki
2015-02-23 16:42                 ` nitin
2013-06-10 21:57             ` Stratos Karafotis
2013-06-10 23:24               ` Rafael J. Wysocki
2013-06-13 21:22                 ` Stratos Karafotis
2013-06-13 21:40                   ` Borislav Petkov
2013-06-13 22:04                     ` Stratos Karafotis
2013-06-13 22:38                       ` Borislav Petkov
2013-06-13 22:15                     ` Rafael J. Wysocki
2013-06-13 22:37                       ` Borislav Petkov
2013-06-14 12:46                         ` Rafael J. Wysocki
2013-06-14 12:44                           ` Borislav Petkov
2013-06-14 12:55                             ` Rafael J. Wysocki
2013-06-14 15:53                               ` Stratos Karafotis
  -- strict thread matches above, loose matches on Subject: below --
2013-06-06 12:56 Stratos Karafotis
2013-06-06 12:54 Stratos Karafotis
2013-06-06 13:15 ` Borislav Petkov
2013-06-05 16:01 Stratos Karafotis
2013-06-05 16:17 ` Borislav Petkov
2013-06-05 16:58   ` David C Niemi
2013-06-06  9:55     ` Borislav Petkov
2013-06-06  9:57       ` Viresh Kumar
2013-06-06 13:50       ` David C Niemi
2013-06-05 17:13   ` Stratos Karafotis
2013-06-05 20:35     ` Rafael J. Wysocki
2013-06-06 10:01       ` Borislav Petkov
2013-06-06 10:10         ` Viresh Kumar
2013-06-06 12:10           ` Borislav Petkov
2013-06-06 16:46             ` Stratos Karafotis
2013-06-06 17:11               ` Borislav Petkov
2013-06-06 17:32                 ` Stratos Karafotis
2013-06-07 19:14       ` Stratos Karafotis
2013-06-07 20:57         ` Rafael J. Wysocki
2013-06-08  9:56           ` Stratos Karafotis
2013-06-08 11:18             ` Rafael J. Wysocki

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox