Re: xen: arm: beginning the removal of mode_switch.S

[Ian Campbell <ian.campbell@citrix.com>]

On Thu, 2013-08-15 at 14:46 +0100, Tim Deegan wrote:
> At 12:51 +0100 on 15 Aug (1376571083), Ian Campbell wrote:
> > At the moment it is sufficient to allow us to do away with the
> > enter_hyp_mode bits and the clock frequency, gic setup etc, along the
> > lines of the patch below.
> > 
> > It doesn't yet allow us to get rid of the kick_cpus stuff. My plan for
> > platforms which don't do the right thing here would be to add a
> > mechanism to use dtb /memreserve/ (and teach Xen about that construct)
> > to carve out a little bit of memory which secondary CPUs could safely be
> > left spinning in. The platform code would expose its normal interface
> > (e.g. SYS_FLAGS on vexpress and fastmodel), eventually maybe we'd do
> > PSCI too (which might let us skip reserving some memory since 2ndary
> > cpus would be in secure mode and could use the special ram regions
> > reserved for that)
> 
> Sorry, I'm not quite clear -- do you plan to have this bootwrapper
> handle the spinning CPUs, with a semihosting callback from Xen to
> release them?  That sounds pretty good.

Yes to spinning, no to semihosting (that's model specific).

On vexpress native this would be handled by writing the start paddr to
SYS_FLAGS and doing a "sev", which will wake up the secondaries (which
are spinning doing wfe) and cause them to jump to that address, so I
would expect the wrapper to implement the same but spinning in NS HYP
mode instead -- in fact the code is there already but I haven't tried it
(bet its broken).

Other platforms have similar mechanisms and I would expect the
bootwrapper to do something similar and mimic whatever the mechanism is
in NS HYP mode.

Thinking about PSCI a bit more, it is likely that if the platform
implements PSCI then it will do it correctly and bring up secondaries in
HYP mode. Probably there would be no possibility of boot-wrapper using
monitor mode anyway just by the nature of PSCI, so we'd be between a
rock and a hard place. I think it'll just work though with no
bootwrapper needed.

> The bootloader looks broadly good to me, including all the kconfig
> stuff.  Do you think we'd maintain that in xen.org or upstream?

I'm not sure, I'm happy to keep it as a local Xen.org hack and when I've
spoken to the upstream in the past they considered it a hack too and
seemed hppy with the idea of us forking. I think they only really care
about models anyway since they don't need to paper over issues on native
so much. Or maybe KVM to care about this. I'll sound them out...

Ian.