Re: [PATCH 2/3] arm64: Consolidate hotplug notifier for instruction emulation

[Will Deacon <will.deacon@arm.com>]

On Fri, Jan 16, 2015 at 04:32:54PM +0000, Mark Rutland wrote:
> On Fri, Jan 16, 2015 at 04:07:30PM +0000, Will Deacon wrote:
> > On Thu, Jan 15, 2015 at 12:36:05PM +0000, Suzuki K. Poulose wrote:
> > > From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
> > > 
> > > As of now each insn_emulation has a cpu hotplug notifier that
> > > enables/disables the CPU feature bit for the functionality. This
> > > patch re-arranges the code, such that there is only one notifier
> > > that runs through the list of registered emulation hooks and runs
> > > their corresponding set_hw_mode.
> > > 
> > > We do nothing when a CPU is dying as we will set the appropriate bits
> > > as it comes back online based on the state of the hooks.
> > > 
> > > Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> > > Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> > > ---
> > >  Documentation/arm64/legacy_instructions.txt |    4 +
> > >  arch/arm64/include/asm/cpufeature.h         |    2 +
> > >  arch/arm64/kernel/armv8_deprecated.c        |  113 +++++++++++++++------------
> > >  3 files changed, 69 insertions(+), 50 deletions(-)
> > > 
> > > diff --git a/Documentation/arm64/legacy_instructions.txt b/Documentation/arm64/legacy_instructions.txt
> > > index a3b3da2..0a4dc26 100644
> > > --- a/Documentation/arm64/legacy_instructions.txt
> > > +++ b/Documentation/arm64/legacy_instructions.txt
> > > @@ -27,6 +27,10 @@ behaviours and the corresponding values of the sysctl nodes -
> > >    instructions. Using hardware execution generally provides better
> > >    performance, but at the loss of ability to gather runtime statistics
> > >    about the use of the deprecated instructions.
> > > +  Note: Emulation of a deprecated instruction depends on the availability
> > > +  of the feature on all the active CPUs. In case of CPU hotplug, if a new
> > > +  CPU doesn't support a feature, it could result in the abortion of the
> > > +  hotplug operation.
> > 
> > Is this true? We should be able to *emulate* the instruction anywhere,
> > it's the "hardware execution" setting that needs CPU support.
> 
> Not quite. In ARMv8 there are three possible cases for endianness
> support:
> 
> (a) Mixed endian support at all levels
>     ID_AA64MMFR0_EL1.BigEnd == 0b0001
>     ID_AA64MMFR0_EL1.BigEnd0 == 0b0000 (RES0, invalid)
> 
> (b) Mixed endian support at EL0 only
>     ID_AA64MMFR0_EL1.BigEnd == 0b0000
>     ID_AA64MMFR0_EL1.BigEnd0 == 0b0001
>     SCTLR_EL1.EE has a fixed value
> 
> (c) No mixed endian support at all
>     ID_AA64MMFR0_EL1.BigEnd == 0b0000
>     ID_AA64MMFR0_EL1.BigEnd0 == 0b0000
>     SCTLR_EL1.{EE,E0E} have fixed values
> 
> We can emulate setend in cases (a) and (b), but not (c) unless we
> trapped every single instruction that could access memory. I don't think
> that's feasible.
> 
> Also fun in case (c) is that the kernel may not be able to run on all
> CPUs (consider a BE kernel where some CPUs are fixed to LE at EL1).
> 
> I hope no-one builds a system where CPUs are mismatched w.r.t.
> endianness support.

Agreed, the problem here is all down to the wording.
Documentation/arm64/legacy_instructions.txt isn't specific to SETEND
emulation and saying "Emulation of a deprecated instruction depends on the
availability of the feature on all the active CPUs." as a blanket statement
doesn't make a lot of sense in isolation. It's certainly not the case for
SWP and CP15 barriers, for example.

With SETEND, we have requirements on mixed-endian support for the emulation.
Fine, but let's not muddy the waters around the general CPU requirements
for deprecated instruction emulation.

Will