Re: [PATCH v16 3/8] cpu/hotplug: Add dynamic parallel bringup states before CPUHP_BRINGUP_CPU

David Woodhouse
2023-03-24T00:07:47Z

[-- Attachment #1: Type: text/plain, Size: 2770 bytes --]

On Fri, 2023-03-24 at 00:48 +0100, Thomas Gleixner wrote:
> On Thu, Mar 23 2023 at 22:49, David Woodhouse wrote:
> > On Thu, 2023-03-23 at 23:36 +0100, Thomas Gleixner wrote:
> > > There is no point in special casing this. All architectures can invoke
> > > the CPUHP_BP_* states before CPUHP_BRINGUP_CPU for each to be brought up
> > > CPU first. So this can be made unconditional and common exercised code.
> > >
> >
> > There were three paragraphs in the commit message explaining why I
> > didn't want to do that. It didn't work for x86 before I started, and I
> > haven't reviewed *every* other architecture to ensure that it will work
> > there. It was opt-in for a reason. :)
>
> I noticed myself before reading your reply :)
>
> > > Aside of that this dynamic state range is pretty pointless. There is
> > > really nothing dynamic here and there is no real good reason to have
> > > four dynamic parallel states just because.
> >
> > The original patch set did use more than one state; the plan to do
> > microcode updates in parallel does involve doing at least one more, I
> > believe.
>
> I don't think so. The micro code muck can completely serialize itself
> and does not need control CPU assistance if done right. If we go there
> we have to fix that mess and not just proliferatng it with moar duct tape.
>
> > > +       /*
> > > +        * Fully per AP serialized bringup from here on. If the
> > > +        * architecture does no register the CPUHP_BP_PARALLEL_STARTUP
> > > +        * state, this step sends the startup IPI first.
> > > +        */
> >
> > Not sure I'd conceded that yet; the APs do their *own* bringup from
> > here, and that really ought to be able to run in parallel.
>
> Somewhere in the distance future once we've
>
>   1) sorted the mandatory synchronization points, e.g. TSC sync in the
>      early bootup phase.

That's why we have four of them... :)

>   2) audited _all_ AP state callbacks that they are able to cope with
>      parallel bringup.
>
>      That's a long road as there are tons of assumptions about the
>      implicit CPU hotplug serialization in those callbacks.
>
>      Just because it did not explode in your face yet does not mean this
>      is safe.
>
>      I just looked at 10 randomly picked AP online callbacks and found 5
>      of them being not ready :)

Oh, it's totally hosed, absolutely. I don't think even my most
ambitious hacks had even tried it yet. But I want to, so I wasn't about
to add a comment saying the opposite.

But it's fine; removing the comment is the *least* of the work to be
done in making that bit actually work in parallel :)

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]