* smpboot seems to be confused on dual EPYC system @ 2018-05-20 13:23 Gabriel C 2018-05-27 20:07 ` Borislav Petkov 0 siblings, 1 reply; 7+ messages in thread From: Gabriel C @ 2018-05-20 13:23 UTC (permalink / raw) To: LKML; +Cc: X86 ML, Thomas Gleixner Hello, I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs. On that box for some reason spmboot things '4' logical packages are possible. Max CPUs can be 128 ( 2 * 32C/64T ), however only 2 CPUs are possible not 4 and therefor there is no way to add other 64 hotplug CPUs. ( not even sure cpu hotplug will work here ) $ dmesg | egrep "smpboot|NR_CPUS|CPUs" [ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:128 nr_cpu_ids:128 nr_node_ids:8 [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=128, Nodes=8 [ 0.000000] RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=128. [ 0.037000] smpboot: CPU0: AMD EPYC 7281 16-Core Processor (family: 0x17, model: 0x1, stepping: 0x2) [ 0.048005] smp: Bringing up secondary CPUs ... [ 0.055042] .... node #0, CPUs: #1 #2 #3 [ 0.079029] .... node #1, CPUs: #4 #5 #6 #7 [ 0.111049] .... node #2, CPUs: #8 #9 #10 #11 [ 0.143050] .... node #3, CPUs: #12 #13 #14 #15 [ 0.175031] .... node #4, CPUs: #16 #17 #18 #19 [ 0.217031] .... node #5, CPUs: #20 #21 #22 #23 [ 0.249062] .... node #6, CPUs: #24 #25 #26 #27 [ 0.281058] .... node #7, CPUs: #28 #29 #30 #31 [ 0.313028] .... node #0, CPUs: #32 #33 #34 #35 [ 0.345028] .... node #1, CPUs: #36 #37 #38 #39 [ 0.377046] .... node #2, CPUs: #40 #41 #42 #43 [ 0.409043] .... node #3, CPUs: #44 #45 #46 #47 [ 0.441028] .... node #4, CPUs: #48 #49 #50 #51 [ 0.473029] .... node #5, CPUs: #52 #53 #54 #55 [ 0.505060] .... node #6, CPUs: #56 #57 #58 #59 [ 0.537052] .... node #7, CPUs: #60 #61 #62 #63 [ 0.562171] smp: Brought up 8 nodes, 64 CPUs [ 0.562171] smpboot: Max logical packages: 4 [ 0.562171] smpboot: Total of 64 processors activated (268089.53 BogoMIPS) Can someone have a look ? Best Regards, Gabriel C ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system 2018-05-20 13:23 smpboot seems to be confused on dual EPYC system Gabriel C @ 2018-05-27 20:07 ` Borislav Petkov [not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com> 0 siblings, 1 reply; 7+ messages in thread From: Borislav Petkov @ 2018-05-27 20:07 UTC (permalink / raw) To: Gabriel C; +Cc: LKML, X86 ML, Thomas Gleixner On Sun, May 20, 2018 at 03:23:32PM +0200, Gabriel C wrote: > Hello, > > I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs. > > On that box for some reason spmboot things '4' logical packages > are possible. First of all, which kernel? > Max CPUs can be 128 ( 2 * 32C/64T ), however only 2 CPUs are possible > not 4 and therefor there is no way to add other 64 hotplug CPUs. > ( not even sure cpu hotplug will work here ) > > $ dmesg | egrep "smpboot|NR_CPUS|CPUs" > [ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs Your BIOS is made to support 32-Core CPUs too. > [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:128 > nr_cpu_ids:128 nr_node_ids:8 > [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=128, Nodes=8 > [ 0.000000] RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=128. > [ 0.037000] smpboot: CPU0: AMD EPYC 7281 16-Core Processor (family: > 0x17, model: 0x1, stepping: 0x2) That says 2 * 16C * 2T = 64T. > [ 0.048005] smp: Bringing up secondary CPUs ... > [ 0.055042] .... node #0, CPUs: #1 #2 #3 > [ 0.079029] .... node #1, CPUs: #4 #5 #6 #7 > [ 0.111049] .... node #2, CPUs: #8 #9 #10 #11 > [ 0.143050] .... node #3, CPUs: #12 #13 #14 #15 > [ 0.175031] .... node #4, CPUs: #16 #17 #18 #19 > [ 0.217031] .... node #5, CPUs: #20 #21 #22 #23 > [ 0.249062] .... node #6, CPUs: #24 #25 #26 #27 > [ 0.281058] .... node #7, CPUs: #28 #29 #30 #31 > [ 0.313028] .... node #0, CPUs: #32 #33 #34 #35 > [ 0.345028] .... node #1, CPUs: #36 #37 #38 #39 > [ 0.377046] .... node #2, CPUs: #40 #41 #42 #43 > [ 0.409043] .... node #3, CPUs: #44 #45 #46 #47 > [ 0.441028] .... node #4, CPUs: #48 #49 #50 #51 > [ 0.473029] .... node #5, CPUs: #52 #53 #54 #55 > [ 0.505060] .... node #6, CPUs: #56 #57 #58 #59 > [ 0.537052] .... node #7, CPUs: #60 #61 #62 #63 > [ 0.562171] smp: Brought up 8 nodes, 64 CPUs > [ 0.562171] smpboot: Max logical packages: 4 > [ 0.562171] smpboot: Total of 64 processors activated (268089.53 BogoMIPS) You have 64 logical CPUs. So what is the problem? -- Regards/Gruss, Boris. Good mailing practices for 400: avoid top-posting and trim the reply. ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com>]
* Re: smpboot seems to be confused on dual EPYC system [not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com> @ 2018-05-27 21:37 ` Gabriel C 2018-05-27 22:51 ` Gabriel C 2018-05-27 23:05 ` Borislav Petkov 2 siblings, 0 replies; 7+ messages in thread From: Gabriel C @ 2018-05-27 21:37 UTC (permalink / raw) To: Borislav Petkov; +Cc: LKML, X86 ML, Thomas Gleixner ( sorry I didn't noticed google changed here something , resending to the list as plain text) 2018-05-27 23:22 GMT+02:00 Gabriel C <nix.or.die@gmail.com>: > > 2018-05-27 22:07 GMT+02:00 Borislav Petkov <bp@alien8.de>: >> >> On Sun, May 20, 2018 at 03:23:32PM +0200, Gabriel C wrote: >> > Hello, >> > >> > I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs. >> > >> > On that box for some reason spmboot things '4' logical packages >> > are possible. >> >> First of all, which kernel? > > > That box runs right now 4.16.11 >> >> >> > Max CPUs can be 128 ( 2 * 32C/64T ), however only 2 CPUs are possible >> > not 4 and therefor there is no way to add other 64 hotplug CPUs. >> > ( not even sure cpu hotplug will work here ) >> > >> > $ dmesg | egrep "smpboot|NR_CPUS|CPUs" >> > [ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs >> >> Your BIOS is made to support 32-Core CPUs too. > > > 128 CPUs are maximal possible however there is no way to have > 2 CPUs fully populated , in this case 64C and have another 64 hotplug CPUs > > That would mean that's an s4 system while impossible with EPYCs. >> >> >> > [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:128 >> > nr_cpu_ids:128 nr_node_ids:8 >> > [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=128, >> > Nodes=8 >> > [ 0.000000] RCU restricting CPUs from NR_CPUS=8192 to >> > nr_cpu_ids=128. >> > [ 0.037000] smpboot: CPU0: AMD EPYC 7281 16-Core Processor (family: >> > 0x17, model: 0x1, stepping: 0x2) >> >> That says 2 * 16C * 2T = 64T. > > > That says whole topology is set to 128 no ? > Eg to max possible but not to what the system runs now. > >> >> >> > [ 0.048005] smp: Bringing up secondary CPUs ... >> > [ 0.055042] .... node #0, CPUs: #1 #2 #3 >> > [ 0.079029] .... node #1, CPUs: #4 #5 #6 #7 >> > [ 0.111049] .... node #2, CPUs: #8 #9 #10 #11 >> > [ 0.143050] .... node #3, CPUs: #12 #13 #14 #15 >> > [ 0.175031] .... node #4, CPUs: #16 #17 #18 #19 >> > [ 0.217031] .... node #5, CPUs: #20 #21 #22 #23 >> > [ 0.249062] .... node #6, CPUs: #24 #25 #26 #27 >> > [ 0.281058] .... node #7, CPUs: #28 #29 #30 #31 >> > [ 0.313028] .... node #0, CPUs: #32 #33 #34 #35 >> > [ 0.345028] .... node #1, CPUs: #36 #37 #38 #39 >> > [ 0.377046] .... node #2, CPUs: #40 #41 #42 #43 >> > [ 0.409043] .... node #3, CPUs: #44 #45 #46 #47 >> > [ 0.441028] .... node #4, CPUs: #48 #49 #50 #51 >> > [ 0.473029] .... node #5, CPUs: #52 #53 #54 #55 >> > [ 0.505060] .... node #6, CPUs: #56 #57 #58 #59 >> > [ 0.537052] .... node #7, CPUs: #60 #61 #62 #63 >> > [ 0.562171] smp: Brought up 8 nodes, 64 CPUs >> > [ 0.562171] smpboot: Max logical packages: 4 >> > [ 0.562171] smpboot: Total of 64 processors activated (268089.53 >> > BogoMIPS) >> >> You have 64 logical CPUs. >> >> So what is the problem? >> > > The problem is that being wrong .. > > >[ 0.562171] smpboot: Max logical packages: 4 > > These are max 2 not 4. > > Regards > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system [not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com> 2018-05-27 21:37 ` Gabriel C @ 2018-05-27 22:51 ` Gabriel C 2018-05-27 23:16 ` Borislav Petkov 2018-05-27 23:05 ` Borislav Petkov 2 siblings, 1 reply; 7+ messages in thread From: Gabriel C @ 2018-05-27 22:51 UTC (permalink / raw) To: Borislav Petkov; +Cc: LKML, X86 ML, Thomas Gleixner 2018-05-27 23:22 GMT+02:00 Gabriel C <nix.or.die@gmail.com>: > > 2018-05-27 22:07 GMT+02:00 Borislav Petkov <bp@alien8.de>: >> >> On Sun, May 20, 2018 at 03:23:32PM +0200, Gabriel C wrote: >> > Hello, >> > >> > I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs. >> > >> > On that box for some reason spmboot things '4' logical packages >> > are possible. >> >> First of all, which kernel? > > Here one box with single socket with an AMD EPYC 7551P on H11SSL-NC mobo running kernel 4.16.10: ... [ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs [ 0.030000] smpboot: CPU0: AMD EPYC 7551P 32-Core Processor (family: 0x17, model: 0x1, stepping: 0x2) [ 0.545084] smpboot: Max logical packages: 2 [ 0.545084] smpboot: Total of 64 processors activated (255989.37 BogoMIPS) ... 128CPUs for sure not possible and for sure no way to have '2' CPUs installed. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system 2018-05-27 22:51 ` Gabriel C @ 2018-05-27 23:16 ` Borislav Petkov 2018-05-27 23:25 ` Gabriel C 0 siblings, 1 reply; 7+ messages in thread From: Borislav Petkov @ 2018-05-27 23:16 UTC (permalink / raw) To: Gabriel C; +Cc: LKML, X86 ML, Thomas Gleixner On Mon, May 28, 2018 at 12:51:17AM +0200, Gabriel C wrote: > 128CPUs for sure not possible and for sure no way to have '2' CPUs installed. With that last "'2' CPUs" you mean, two physical processors and thus two sockets, right? In any case, this info is what we've gotten from the BIOS so if that BIOS tells us this way... Again, you don't see any other issues besides the misrepresenting of max packages or? -- Regards/Gruss, Boris. Good mailing practices for 400: avoid top-posting and trim the reply. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system 2018-05-27 23:16 ` Borislav Petkov @ 2018-05-27 23:25 ` Gabriel C 0 siblings, 0 replies; 7+ messages in thread From: Gabriel C @ 2018-05-27 23:25 UTC (permalink / raw) To: Borislav Petkov; +Cc: LKML, X86 ML, Thomas Gleixner 2018-05-28 1:16 GMT+02:00 Borislav Petkov <bp@alien8.de>: > On Mon, May 28, 2018 at 12:51:17AM +0200, Gabriel C wrote: >> 128CPUs for sure not possible and for sure no way to have '2' CPUs installed. > > With that last "'2' CPUs" you mean, two physical processors and thus two > sockets, right? Yes. > > In any case, this info is what we've gotten from the BIOS so if that > BIOS tells us this way... Sure .. not the first time BIOSes lies about things :) I never said that being an kernel bug I just noticed something is getting confused. > > Again, you don't see any other issues besides the misrepresenting of max > packages or? > Right no other issues. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system [not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com> 2018-05-27 21:37 ` Gabriel C 2018-05-27 22:51 ` Gabriel C @ 2018-05-27 23:05 ` Borislav Petkov 2 siblings, 0 replies; 7+ messages in thread From: Borislav Petkov @ 2018-05-27 23:05 UTC (permalink / raw) To: Gabriel C; +Cc: LKML, X86 ML, Thomas Gleixner On Sun, May 27, 2018 at 11:22:18PM +0200, Gabriel C wrote: > The problem is that being wrong .. > > >[ 0.562171] smpboot: Max logical packages: 4 > > These are max 2 not 4. So the whole "problem" is that it miscounts max packages. Which means we waste a couple of KBs of memory, tops. Any other, real problems you're experiencing? -- Regards/Gruss, Boris. Good mailing practices for 400: avoid top-posting and trim the reply. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-05-27 23:26 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-05-20 13:23 smpboot seems to be confused on dual EPYC system Gabriel C
2018-05-27 20:07 ` Borislav Petkov
[not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com>
2018-05-27 21:37 ` Gabriel C
2018-05-27 22:51 ` Gabriel C
2018-05-27 23:16 ` Borislav Petkov
2018-05-27 23:25 ` Gabriel C
2018-05-27 23:05 ` Borislav Petkov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox