Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

Len Brown
2015-05-12T08:03:39Z

On Thu, May 7, 2015 at 6:23 AM, Ingo Molnar <mingo@kernel.org> wrote:

These numbers do not look right.

> Here's what the boot time looks like on a 120 CPUs system, with the
> patch applied:
>
> [ 0.558947] x86: Booting SMP configuration:
> [ 0.563375] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14
> [ 0.644851] .... node #1, CPUs: #15 #16 #17 #18 #19 #20 #21 #22 #23 #24 #25 #26 #27 #28 #29
> [ 0.830474] .... node #2, CPUs: #30 #31 #32 #33 #34 #35 #36 #37 #38 #39 #40 #41 #42 #43 #44
> [ 1.016357] .... node #3, CPUs: #45 #46 #47 #48 #49 #50 #51 #52 #53 #54 #55 #56 #57 #58 #59
> [ 1.202342] .... node #0, CPUs: #60 #61 #62 #63 #64 #65 #66 #67 #68 #69 #70 #71 #72 #73 #74
> [ 1.283864] .... node #1, CPUs: #75 #76 #77 #78 #79 #80 #81 #82 #83 #84 #85 #86 #87 #88 #89
> [ 1.397131] .... node #2, CPUs: #90 #91 #92 #93 #94 #95 #96 #97 #98 #99 #100 #101 #102 #103 #104
> [ 1.510417] .... node #3, CPUs: #105 #106 #107 #108 #109 #110 #111 #112 #113 #114 #115 #116 #117 #118 #119
> [ 1.620967] x86: Booted up 4 nodes, 120 CPUs
> [ 1.625928] smpboot: Total of 120 processors activated (672866.16 BogoMIPS)
>
> 1.1 seconds to boot 120 CPUs, 10.8 seconds to hit init, that's an
> entirely reasonable runtime I think.
>
> It was 20+ seconds before that, 10+ seconds for the SMP bootup
> sequence.

(1.625928-0.558947) = 1.07 seconds to online 119 additional cpus.
/119 = .0089662268 each, lets call it 9ms.

Here is my ivb-ex running stock fedora 21's Linux-3.19 (no patch applied):

[ 0.404369] x86: Booting SMP configuration:
[ 0.409492] .... node #0, CPUs: #1
[ 0.439900] NMI watchdog: enabled on all CPUs, permanently consumes
one hw-PMU counter.
[ 0.450533] #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14
[ 0.652409] .... node #1, CPUs: #15 #16 #17 #18 #19 #20
#21 #22 #23 #24 #25 #26 #27 #28 #29
[ 0.999645] .... node #2, CPUs: #30 #31 #32 #33 #34 #35
#36 #37 #38 #39 #40 #41 #42 #43 #44
[ 1.346991] .... node #3, CPUs: #45 #46 #47 #48 #49 #50
#51 #52 #53 #54 #55 #56 #57 #58 #59
[ 1.694171] .... node #0, CPUs: #60 #61 #62 #63 #64 #65
#66 #67 #68 #69 #70 #71 #72 #73 #74
[ 1.928248] .... node #1, CPUs: #75 #76 #77 #78 #79 #80
#81 #82 #83 #84 #85 #86 #87 #88 #89
[ 2.198370] .... node #2, CPUs: #90 #91 #92 #93 #94 #95
#96 #97 #98 #99 #100 #101 #102 #103 #104
[ 2.468574] .... node #3, CPUs: #105 #106 #107 #108 #109 #110
#111 #112 #113 #114 #115 #116 #117 #118 #119
[ 2.737884] x86: Booted up 4 nodes, 120 CPUs
[ 2.743758] smpboot: Total of 120 processors activated (671097.18 BogoMIPS)

(2.743758-0.404369) = 2.339389 for all 119 processors
/119 = .01965873109243697478 - lets call it 19ms each

so this baseline case is 19ms/processor -- which matches above, where
we delete 10ms/processor.
But even at 19ms each, this is totals only 2.3 seconds for all 119 processors.
So I don't understand your reference to 10+ seconds, above.

thanks,
Len Brown, Intel Open Source Technology Center

ps. I wish the BIOS boot time on this box were as fast as Linux:-)