Re: [PATCH] x86: Reduce clock calibration time during slave cpu startup

[Andrew Morton <akpm@linux-foundation.org>]

On Wed, 27 Jul 2011 08:57:31 -0500
Jack Steiner <steiner@sgi.com> wrote:

> Reduce the startup time for slave cpus.
> 
> This patch adds hooks for an arch-specific function for clock calibration.
> These hooks are used on x86. They assume all cores in a physical socket
> run at the same core speed. If a newly started cpu has the same phys_proc_id
> as a core already active, use the already-calculated value of loops_per_jiffy.
> 
> This patch reduces the time required to start slave cpus on a 4096 cpu
> system from:
> 	465 sec  OLD
> 	 62 sec NEW

Eight minutes is just stupid.

100ms/cpu is just stupid too.  What's the CPU doing?  Spinning around
counting ticks?  That's parallelizable.

> This reduces boot time on a 4096p system by almost 7 minutes.  Nice...
> 
> 
> Signed-off-by: Jack Steiner <steiner@sgi.com>
> 
> 
> ---
> Note: patch assumes that all multi-core x86 processor sockets have the same
> clock frequency for all cores. AFAIK, this is true & will continue
> to be true for a long time. Have I overlooked anything???

Well, Andi thinks this may become untrue relatively soon.  Then what do
we do?

>  /*
> + * Check if another cpu is in the same socket and has already been calibrated.
> + * If found, use the previous value. This assumes all cores in the same physical
> + * socket have the same core frequency.
> + */
> +unsigned long __cpuinit calibrate_delay_is_known(void)
> +{
> +	int i, cpu = smp_processor_id();
> +
> +	for_each_online_cpu(i)
> +		if (cpu_data(i).phys_proc_id == cpu_data(cpu).phys_proc_id)

This will always match when `i' reaches `cpu'.  Or is this cpu not
online at this time?

> +			return cpu_data(i).loops_per_jiffy;
> +	return 0;
> +}
> +
> +/*
>   * Activate a secondary processor.
>   */
>  notrace static void __cpuinit start_secondary(void *unused)
> Index: linux/init/calibrate.c
> ===================================================================
> --- linux.orig/init/calibrate.c	2011-07-26 08:01:15.571979739 -0500
> +++ linux/init/calibrate.c	2011-07-27 08:39:35.691983745 -0500
> @@ -243,6 +243,20 @@ recalibrate:
>  	return lpj;
>  }
>  
> +/*
> + * Check if cpu calibration delay is already known. For example,
> + * some processors with multi-core sockets may have all sockets
> + * use the same core frequency. It is not necessary to calibrate
> + * each core.
> + *
> + * Architectures should override this function if a faster calibration
> + * method is available.
> + */
> +unsigned long __attribute__((weak)) __cpuinit calibrate_delay_is_known(void)

__weak

> +{
> +	return 0;
> +}
> +
>  void __cpuinit calibrate_delay(void)
>  {
>  	unsigned long lpj;
> @@ -257,6 +271,8 @@ void __cpuinit calibrate_delay(void)
>  		lpj = lpj_fine;
>  		pr_info("Calibrating delay loop (skipped), "
>  			"value calculated using timer frequency.. ");
> +	} else if ((lpj = calibrate_delay_is_known())) {
> +		;
>  	} else if ((lpj = calibrate_delay_direct()) != 0) {
>  		if (!printed)
>  			pr_info("Calibrating delay using timer "