[PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Andrea della Porta]

Aarch32 compatibility mode is enabled at compile time through
CONFIG_COMPAT Kconfig option. This patchset lets 32-bit support
(for both processes and syscalls) be enabled at boot time using
a kernel parameter. Also, it provides a mean for distributions 
to set their own default without sacrificing compatibility support,
that is users can override default behaviour through the kernel
parameter.

*** Notes about syscall management ***
VBAR_EL1 register, which holds the exception table address,
is setup very early in the boot process, before parse_early_param().
This means that it's not possible to access boot parameter before
setting the register. Also, setting the aforementioned register
for secondary cpus is done later in the boot flow.
Several ways to work around this has been considered, among which:

* resetting VBAR_EL1 to point to one of two vector tables (the
  former with 32-bit exceptions handler enabled and the latter
  pointing to unhandled stub, just as if CONFIG_COMPAT is enabled)
  depending on the proposed boot parameter. This has the disadvantage
  to produce a somewhat messy patchset involving several lines,
  has higher cognitive load since there are at least three places
  where the register is getting changed (not near to each other),
  and have implications on other code segments (namely kpti, kvm
  and vdso), requiring special care.

* patching the vector table contents once the early param is available.
  This has most of the implications of the previous option
  (except maybe not impacting other code segments), plus it sounds
  a little 'hackish'.

The chosen approach involves conditional executing 32-bit syscalls
depending on the parameter value. This of course results in a
little performance loss, but has the following advantages:

* all the cons from previously explained alternatives are solved
* users of 32-bit apps on 64-bit kernel are already suffering from
  performance losses due to 32-bit apps not fully leveraging the 64-bit
  processor, so they are already aware of this
* users of 32-bit apps on 64-bit kernel are believed
  to be a minority and most of the time there are sources available
  to be recompiled for 64-bit as a workaround for better performance

It worth mentioning that users of 64-bit apps are, of course,
unaffected.

Based on the work from Nikolay Borisov, see:
Link: https://lkml.org/lkml/2023/6/23/387

Andrea della Porta (4):
  arm64: Introduce aarch32_enabled()
  arm64/process: Make loading of 32bit processes depend on
    aarch32_enabled()
  arm64/entry-common: Make Aarch32 syscalls' availability depend on
    aarch32_enabled()
  arm64: Make Aarch32 emulation boot time configurable

 .../admin-guide/kernel-parameters.txt         |  7 ++++
 arch/arm64/Kconfig                            |  9 +++++
 arch/arm64/include/asm/compat.h               | 12 +++++++
 arch/arm64/kernel/entry-common.c              | 33 +++++++++++++++++--
 arch/arm64/kernel/process.c                   |  2 +-
 5 files changed, 59 insertions(+), 4 deletions(-)

-- 
2.35.3


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 1/4] arm64: Introduce aarch32_enabled()

[Andrea della Porta]

Aarch32 bit support on 64bit kernels depends on whether CONFIG_COMPAT
is selected or not. As it is a compile time option it doesn't
provide the flexibility to have distributions set their own policy for
Aarch32 support and give the user the flexibility to override it.

As a first step introduce aarch32_enabled() which abstracts whether 32
bit compat is turned on or off. Upcoming patches will implement
the ability to set Aarch32 compat state at boot time.

Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
---
 arch/arm64/include/asm/compat.h  | 12 ++++++++++++
 arch/arm64/kernel/entry-common.c |  2 ++
 2 files changed, 14 insertions(+)

diff --git a/arch/arm64/include/asm/compat.h b/arch/arm64/include/asm/compat.h
index ae904a1ad529..f5ef71bd0024 100644
--- a/arch/arm64/include/asm/compat.h
+++ b/arch/arm64/include/asm/compat.h
@@ -98,6 +98,13 @@ static inline int is_compat_thread(struct thread_info *thread)
 
 long compat_arm_syscall(struct pt_regs *regs, int scno);
 
+extern bool __aarch32_enabled;
+
+static inline bool aarch32_enabled(void)
+{
+	return __aarch32_enabled;
+}
+
 #else /* !CONFIG_COMPAT */
 
 static inline int is_compat_thread(struct thread_info *thread)
@@ -105,5 +112,10 @@ static inline int is_compat_thread(struct thread_info *thread)
 	return 0;
 }
 
+static inline bool aarch32_enabled(void)
+{
+	return false;
+}
+
 #endif /* CONFIG_COMPAT */
 #endif /* __ASM_COMPAT_H */
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 0fc94207e69a..69ff9b8c0bde 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -877,6 +877,8 @@ asmlinkage void noinstr el0t_32_error_handler(struct pt_regs *regs)
 {
 	__el0_error_handler_common(regs);
 }
+
+bool __aarch32_enabled __ro_after_init = true;
 #else /* CONFIG_COMPAT */
 UNHANDLED(el0t, 32, sync)
 UNHANDLED(el0t, 32, irq)
-- 
2.35.3


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 2/4] arm64/process: Make loading of 32bit processes depend on aarch32_enabled()

[Andrea della Porta]

Major aspect of Aarch32 emulation is the ability to load 32bit
processes.
That's currently decided (among others) by compat_elf_check_arch().

Make the macro use aarch32_enabled() to decide if Aarch32 compat is
enabled before loading a 32bit process.

Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
---
 arch/arm64/kernel/process.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 657ea273c0f9..96832f1ec3ee 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -601,7 +601,7 @@ unsigned long arch_align_stack(unsigned long sp)
 #ifdef CONFIG_COMPAT
 int compat_elf_check_arch(const struct elf32_hdr *hdr)
 {
-	if (!system_supports_32bit_el0())
+	if (!system_supports_32bit_el0() || !aarch32_enabled())
 		return false;
 
 	if ((hdr)->e_machine != EM_ARM)
-- 
2.35.3


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 2/4] arm64/process: Make loading of 32bit processes depend on aarch32_enabled()

[Mark Rutland]

On Wed, Oct 18, 2023 at 01:13:20PM +0200, Andrea della Porta wrote:
> Major aspect of Aarch32 emulation is the ability to load 32bit
> processes.
> That's currently decided (among others) by compat_elf_check_arch().
> 
> Make the macro use aarch32_enabled() to decide if Aarch32 compat is
> enabled before loading a 32bit process.
> 
> Signed-off-by: Andrea della Porta <andrea.porta@suse.com>

Why can't you make system_supports_32bit_el0() take the option into account
instead?

Mark.

> ---
>  arch/arm64/kernel/process.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
> index 657ea273c0f9..96832f1ec3ee 100644
> --- a/arch/arm64/kernel/process.c
> +++ b/arch/arm64/kernel/process.c
> @@ -601,7 +601,7 @@ unsigned long arch_align_stack(unsigned long sp)
>  #ifdef CONFIG_COMPAT
>  int compat_elf_check_arch(const struct elf32_hdr *hdr)
>  {
> -	if (!system_supports_32bit_el0())
> +	if (!system_supports_32bit_el0() || !aarch32_enabled())
>  		return false;
>  
>  	if ((hdr)->e_machine != EM_ARM)
> -- 
> 2.35.3
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 2/4] arm64/process: Make loading of 32bit processes depend on aarch32_enabled()

[Andrea della Porta]

On 13:52 Wed 18 Oct     , Mark Rutland wrote:
> On Wed, Oct 18, 2023 at 01:13:20PM +0200, Andrea della Porta wrote:
> > Major aspect of Aarch32 emulation is the ability to load 32bit
> > processes.
> > That's currently decided (among others) by compat_elf_check_arch().
> > 
> > Make the macro use aarch32_enabled() to decide if Aarch32 compat is
> > enabled before loading a 32bit process.
> > 
> > Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> 
> Why can't you make system_supports_32bit_el0() take the option into account
> instead?
>

I may be wrong here, but it seems to me that system_supports_32bit_el0()
answers teh question "can this system supports compat execution?" rather than
"do I want this system to run any compat execution?". That's the point of
aarch32_enabled(), to state whether we want teh system to run A32 code or not,
regardless of the system supporting it (of course, if the system does not
support A32 in EL0, this is a no-no, but that's another story).

Andrea 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 2/4] arm64/process: Make loading of 32bit processes depend on aarch32_enabled()

[Mark Rutland]

On Thu, Oct 19, 2023 at 02:38:32PM +0200, Andrea della Porta wrote:
> On 13:52 Wed 18 Oct     , Mark Rutland wrote:
> > On Wed, Oct 18, 2023 at 01:13:20PM +0200, Andrea della Porta wrote:
> > > Major aspect of Aarch32 emulation is the ability to load 32bit
> > > processes.
> > > That's currently decided (among others) by compat_elf_check_arch().
> > > 
> > > Make the macro use aarch32_enabled() to decide if Aarch32 compat is
> > > enabled before loading a 32bit process.
> > > 
> > > Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> > 
> > Why can't you make system_supports_32bit_el0() take the option into account
> > instead?
> >
> 
> I may be wrong here, but it seems to me that system_supports_32bit_el0()
> answers teh question "can this system supports compat execution?" rather than
> "do I want this system to run any compat execution?". That's the point of
> aarch32_enabled(), to state whether we want teh system to run A32 code or not,
> regardless of the system supporting it (of course, if the system does not
> support A32 in EL0, this is a no-no, but that's another story).

That's what the implementation does today, but we're really using it as a "do
we intend for 32-bit EL0 to work?" predicate, and generally the
system_supports_${FEATURE}() helpers are affected by the combination of actual
HW support, kernel config options, *and* kernel command line options. For
example, system_supports_sve() is affected by both CONFIG_ARM64_SVE and the
"arm64.nosve" command line option.

Thanks,
Mark.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 2/4] arm64/process: Make loading of 32bit processes depend on aarch32_enabled()

[Andrea della Porta]

On 15:27 Thu 19 Oct     , Mark Rutland wrote:
> On Thu, Oct 19, 2023 at 02:38:32PM +0200, Andrea della Porta wrote:
> > On 13:52 Wed 18 Oct     , Mark Rutland wrote:
> > > On Wed, Oct 18, 2023 at 01:13:20PM +0200, Andrea della Porta wrote:
> > > > Major aspect of Aarch32 emulation is the ability to load 32bit
> > > > processes.
> > > > That's currently decided (among others) by compat_elf_check_arch().
> > > > 
> > > > Make the macro use aarch32_enabled() to decide if Aarch32 compat is
> > > > enabled before loading a 32bit process.
> > > > 
> > > > Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> > > 
> > > Why can't you make system_supports_32bit_el0() take the option into account
> > > instead?
> > >
> > 
> > I may be wrong here, but it seems to me that system_supports_32bit_el0()
> > answers teh question "can this system supports compat execution?" rather than
> > "do I want this system to run any compat execution?". That's the point of
> > aarch32_enabled(), to state whether we want teh system to run A32 code or not,
> > regardless of the system supporting it (of course, if the system does not
> > support A32 in EL0, this is a no-no, but that's another story).
> 
> That's what the implementation does today, but we're really using it as a "do
> we intend for 32-bit EL0 to work?" predicate, and generally the
> system_supports_${FEATURE}() helpers are affected by the combination of actual
> HW support, kernel config options, *and* kernel command line options. For
> example, system_supports_sve() is affected by both CONFIG_ARM64_SVE and the
> "arm64.nosve" command line option.
> 
> Thanks,
> Mark.

Many thanks for the explanation, then inserting aach32_enabled() in
system_supports_32bit_el0() is the way to go.

Andrea

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 2/4] arm64/process: Make loading of 32bit processes depend on aarch32_enabled()

[Mark Rutland, Will Deacon]

On Thu, Oct 19, 2023 at 04:32:27PM +0200, Andrea della Porta wrote:
> On 15:27 Thu 19 Oct     , Mark Rutland wrote:
> > On Thu, Oct 19, 2023 at 02:38:32PM +0200, Andrea della Porta wrote:
> > > On 13:52 Wed 18 Oct     , Mark Rutland wrote:
> > > > On Wed, Oct 18, 2023 at 01:13:20PM +0200, Andrea della Porta wrote:
> > > > > Major aspect of Aarch32 emulation is the ability to load 32bit
> > > > > processes.
> > > > > That's currently decided (among others) by compat_elf_check_arch().
> > > > > 
> > > > > Make the macro use aarch32_enabled() to decide if Aarch32 compat is
> > > > > enabled before loading a 32bit process.
> > > > > 
> > > > > Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> > > > 
> > > > Why can't you make system_supports_32bit_el0() take the option into account
> > > > instead?
> > > >
> > > 
> > > I may be wrong here, but it seems to me that system_supports_32bit_el0()
> > > answers teh question "can this system supports compat execution?" rather than
> > > "do I want this system to run any compat execution?". That's the point of
> > > aarch32_enabled(), to state whether we want teh system to run A32 code or not,
> > > regardless of the system supporting it (of course, if the system does not
> > > support A32 in EL0, this is a no-no, but that's another story).
> > 
> > That's what the implementation does today, but we're really using it as a "do
> > we intend for 32-bit EL0 to work?" predicate, and generally the
> > system_supports_${FEATURE}() helpers are affected by the combination of actual
> > HW support, kernel config options, *and* kernel command line options. For
> > example, system_supports_sve() is affected by both CONFIG_ARM64_SVE and the
> > "arm64.nosve" command line option.
> > 
> > Thanks,
> > Mark.
> 
> Many thanks for the explanation, then inserting aach32_enabled() in
> system_supports_32bit_el0() is the way to go.

I think what we should do here is clean up the way we implement
system_supports_32bit_el0() such that it can be a cpucap, and have the
conditions that would affect aarch32_enabled() feed into that. That way,
system_supports_32bit_el0() will compile down to a single branch/nop (or elided
entirely when known to be false at compile-time), and with that I think can
reasonably fold the existing UNHANDLED() logic into the entry-common.c
exception handlers as a simplification.

The only obviously painful part is that enable_mismatched_32bit_el0() allows
(mismatched) AArch32 support to be enabled after we finalize system cpucaps, as
part of a late hotplug. I suspect that was implemented that way for expedience
rather than because we wanted to enable mismatched AArch32 after finalizing
cpucaps.

Will, do you remember why we used a cpuhp callback for enabling mismatched
32-bit support? I couldn't see anything explicit in the commit message for:

  2122a833316f2f3f ("arm64: Allow mismatched 32-bit EL0 support")

... and I suspect it was just easier to write that way, rather than adding more
code around setup_system_capabilities() ?

Mark.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 3/4] arm64/entry-common: Make Aarch32 syscalls' availability depend on aarch32_enabled()

[Andrea della Porta]

Another major aspect of supporting running of 32bit processes is the
ability to access 32bit syscalls. Such syscalls can be invoked by
using the svc instruction.

If Aarch32 emulation is disabled ensure that calling svc results
in the same behavior as if CONFIG_COMPAT has not been enabled (i.e.
a kernel panic).

Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
---
 arch/arm64/kernel/entry-common.c | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 69ff9b8c0bde..32761760d9dd 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -802,6 +802,11 @@ asmlinkage void noinstr el0t_64_error_handler(struct pt_regs *regs)
 }
 
 #ifdef CONFIG_COMPAT
+UNHANDLED(el0t, 32, sync_ni)
+UNHANDLED(el0t, 32, irq_ni)
+UNHANDLED(el0t, 32, fiq_ni)
+UNHANDLED(el0t, 32, error_ni)
+
 static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr)
 {
 	enter_from_user_mode(regs);
@@ -821,6 +826,11 @@ static void noinstr el0_svc_compat(struct pt_regs *regs)
 
 asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs)
 {
+	if (!aarch32_enabled()) {
+		el0t_32_sync_ni_handler(regs);
+		return;
+	}
+
 	unsigned long esr = read_sysreg(esr_el1);
 
 	switch (ESR_ELx_EC(esr)) {
@@ -865,17 +875,26 @@ asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs)
 
 asmlinkage void noinstr el0t_32_irq_handler(struct pt_regs *regs)
 {
-	__el0_irq_handler_common(regs);
+	if (!aarch32_enabled())
+		el0t_32_irq_ni_handler(regs);
+	else
+		__el0_irq_handler_common(regs);
 }
 
 asmlinkage void noinstr el0t_32_fiq_handler(struct pt_regs *regs)
 {
-	__el0_fiq_handler_common(regs);
+	if (!aarch32_enabled())
+		el0t_32_fiq_ni_handler(regs);
+	else
+		__el0_fiq_handler_common(regs);
 }
 
 asmlinkage void noinstr el0t_32_error_handler(struct pt_regs *regs)
 {
-	__el0_error_handler_common(regs);
+	if (!aarch32_enabled())
+		el0t_32_error_ni_handler(regs);
+	else
+		__el0_error_handler_common(regs);
 }
 
 bool __aarch32_enabled __ro_after_init = true;
-- 
2.35.3


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/4] arm64/entry-common: Make Aarch32 syscalls' availability depend on aarch32_enabled()

[Mark Rutland]

On Wed, Oct 18, 2023 at 01:13:21PM +0200, Andrea della Porta wrote:
> Another major aspect of supporting running of 32bit processes is the
> ability to access 32bit syscalls. Such syscalls can be invoked by
> using the svc instruction.
> 
> If Aarch32 emulation is disabled ensure that calling svc results
> in the same behavior as if CONFIG_COMPAT has not been enabled (i.e.
> a kernel panic).

It's not "emulation" it's directly supported by the hardware.

> 
> Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> ---
>  arch/arm64/kernel/entry-common.c | 25 ++++++++++++++++++++++---
>  1 file changed, 22 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
> index 69ff9b8c0bde..32761760d9dd 100644
> --- a/arch/arm64/kernel/entry-common.c
> +++ b/arch/arm64/kernel/entry-common.c
> @@ -802,6 +802,11 @@ asmlinkage void noinstr el0t_64_error_handler(struct pt_regs *regs)
>  }
>  
>  #ifdef CONFIG_COMPAT
> +UNHANDLED(el0t, 32, sync_ni)
> +UNHANDLED(el0t, 32, irq_ni)
> +UNHANDLED(el0t, 32, fiq_ni)
> +UNHANDLED(el0t, 32, error_ni)

IRQ, FIQ, and SError are not syscalls, so the commit title is bad.

> +
>  static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr)
>  {
>  	enter_from_user_mode(regs);
> @@ -821,6 +826,11 @@ static void noinstr el0_svc_compat(struct pt_regs *regs)
>  
>  asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs)
>  {
> +	if (!aarch32_enabled()) {
> +		el0t_32_sync_ni_handler(regs);
> +		return;
> +	}

Why do we have to do this at all?

If we don't have AArch32 tasks, these paths are unreachable. Why do we need to
check that they aren't called?

Mark.

> +
>  	unsigned long esr = read_sysreg(esr_el1);
>  
>  	switch (ESR_ELx_EC(esr)) {
> @@ -865,17 +875,26 @@ asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs)
>  
>  asmlinkage void noinstr el0t_32_irq_handler(struct pt_regs *regs)
>  {
> -	__el0_irq_handler_common(regs);
> +	if (!aarch32_enabled())
> +		el0t_32_irq_ni_handler(regs);
> +	else
> +		__el0_irq_handler_common(regs);
>  }
>  
>  asmlinkage void noinstr el0t_32_fiq_handler(struct pt_regs *regs)
>  {
> -	__el0_fiq_handler_common(regs);
> +	if (!aarch32_enabled())
> +		el0t_32_fiq_ni_handler(regs);
> +	else
> +		__el0_fiq_handler_common(regs);
>  }
>  
>  asmlinkage void noinstr el0t_32_error_handler(struct pt_regs *regs)
>  {
> -	__el0_error_handler_common(regs);
> +	if (!aarch32_enabled())
> +		el0t_32_error_ni_handler(regs);
> +	else
> +		__el0_error_handler_common(regs);
>  }
>  
>  bool __aarch32_enabled __ro_after_init = true;
> -- 
> 2.35.3
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/4] arm64/entry-common: Make Aarch32 syscalls' availability depend on aarch32_enabled()

[Andrea della Porta]

On 13:57 Wed 18 Oct     , Mark Rutland wrote:
> On Wed, Oct 18, 2023 at 01:13:21PM +0200, Andrea della Porta wrote:
> > Another major aspect of supporting running of 32bit processes is the
> > ability to access 32bit syscalls. Such syscalls can be invoked by
> > using the svc instruction.
> > 
> > If Aarch32 emulation is disabled ensure that calling svc results
> > in the same behavior as if CONFIG_COMPAT has not been enabled (i.e.
> > a kernel panic).
> 
> It's not "emulation" it's directly supported by the hardware.

You're right. I also struggled to use this label but I just reused the same
name from Nikolai's patchset for x86, in the hope that the option would be
more recognizable (something like 'ARCH_emulation' that could be used maybe
for other platforms as well), but I agree with you that this is highly
misleading. I will change it to something more straightforward.

> 
> > 
> > Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> > ---
> >  arch/arm64/kernel/entry-common.c | 25 ++++++++++++++++++++++---
> >  1 file changed, 22 insertions(+), 3 deletions(-)
> > 
> > diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
> > index 69ff9b8c0bde..32761760d9dd 100644
> > --- a/arch/arm64/kernel/entry-common.c
> > +++ b/arch/arm64/kernel/entry-common.c
> > @@ -802,6 +802,11 @@ asmlinkage void noinstr el0t_64_error_handler(struct pt_regs *regs)
> >  }
> >  
> >  #ifdef CONFIG_COMPAT
> > +UNHANDLED(el0t, 32, sync_ni)
> > +UNHANDLED(el0t, 32, irq_ni)
> > +UNHANDLED(el0t, 32, fiq_ni)
> > +UNHANDLED(el0t, 32, error_ni)
> 
> IRQ, FIQ, and SError are not syscalls, so the commit title is bad.

Agreed. I'll call them exceptions.

> 
> > +
> >  static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr)
> >  {
> >  	enter_from_user_mode(regs);
> > @@ -821,6 +826,11 @@ static void noinstr el0_svc_compat(struct pt_regs *regs)
> >  
> >  asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs)
> >  {
> > +	if (!aarch32_enabled()) {
> > +		el0t_32_sync_ni_handler(regs);
> > +		return;
> > +	}
> 
> Why do we have to do this at all?
> 
> If we don't have AArch32 tasks, these paths are unreachable. Why do we need to
> check that they aren't called?
> 
> Mark.

Agreed. Please see also my previous comments here:
https://lore.kernel.org/lkml/ZTEKabxNdegsbxyv@apocalypse/
https://lore.kernel.org/lkml/ZTD0DAes-J-YQ2eu@apocalypse/

but again, that's only speculative as of now, so we can ignore that part.

Andrea

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/4] arm64/entry-common: Make Aarch32 syscalls' availability depend on aarch32_enabled()

[kernel test robot]

Hi Andrea,

kernel test robot noticed the following build warnings:

[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on arm-perf/for-next/perf arm/for-next arm/fixes kvmarm/next soc/for-next linus/master v6.6-rc6 next-20231020]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Andrea-della-Porta/arm64-Introduce-aarch32_enabled/20231018-191517
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link:    https://lore.kernel.org/r/88bdea628a13747bff32c0c3055d6d6ef7264d96.1697614386.git.andrea.porta%40suse.com
patch subject: [PATCH 3/4] arm64/entry-common: Make Aarch32 syscalls' availability depend on aarch32_enabled()
config: arm64-randconfig-003-20231023 (https://download.01.org/0day-ci/archive/20231023/202310230423.r2U4Lqr8-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231023/202310230423.r2U4Lqr8-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202310230423.r2U4Lqr8-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> arch/arm64/kernel/entry-common.c:805:11: warning: no previous prototype for 'el0t_32_sync_ni_handler' [-Wmissing-prototypes]
     805 | UNHANDLED(el0t, 32, sync_ni)
         |           ^~~~
   arch/arm64/kernel/entry-common.c:302:25: note: in definition of macro 'UNHANDLED'
     302 | asmlinkage void noinstr el##_##regsize##_##vector##_handler(struct pt_regs *regs)       \
         |                         ^~
>> arch/arm64/kernel/entry-common.c:806:11: warning: no previous prototype for 'el0t_32_irq_ni_handler' [-Wmissing-prototypes]
     806 | UNHANDLED(el0t, 32, irq_ni)
         |           ^~~~
   arch/arm64/kernel/entry-common.c:302:25: note: in definition of macro 'UNHANDLED'
     302 | asmlinkage void noinstr el##_##regsize##_##vector##_handler(struct pt_regs *regs)       \
         |                         ^~
>> arch/arm64/kernel/entry-common.c:807:11: warning: no previous prototype for 'el0t_32_fiq_ni_handler' [-Wmissing-prototypes]
     807 | UNHANDLED(el0t, 32, fiq_ni)
         |           ^~~~
   arch/arm64/kernel/entry-common.c:302:25: note: in definition of macro 'UNHANDLED'
     302 | asmlinkage void noinstr el##_##regsize##_##vector##_handler(struct pt_regs *regs)       \
         |                         ^~
>> arch/arm64/kernel/entry-common.c:808:11: warning: no previous prototype for 'el0t_32_error_ni_handler' [-Wmissing-prototypes]
     808 | UNHANDLED(el0t, 32, error_ni)
         |           ^~~~
   arch/arm64/kernel/entry-common.c:302:25: note: in definition of macro 'UNHANDLED'
     302 | asmlinkage void noinstr el##_##regsize##_##vector##_handler(struct pt_regs *regs)       \
         |                         ^~


vim +/el0t_32_sync_ni_handler +805 arch/arm64/kernel/entry-common.c

   803	
   804	#ifdef CONFIG_COMPAT
 > 805	UNHANDLED(el0t, 32, sync_ni)
 > 806	UNHANDLED(el0t, 32, irq_ni)
 > 807	UNHANDLED(el0t, 32, fiq_ni)
 > 808	UNHANDLED(el0t, 32, error_ni)
   809	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 4/4] arm64: Make Aarch32 emulation boot time configurable

[Andrea della Porta]

Distributions would like to reduce their attack surface as much as
possible but at the same time they'd want to retain flexibility to
cater to a variety of legacy software. This stems from the conjecture
that compat layer is likely rarely tested and could have latent
security bugs. Ideally distributions will set their default policy
and also give users the ability to override it as appropriate.

To enable this use case, introduce CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED
compile time option, which controls whether 32bit processes/syscalls
should be allowed or not. This option is aimed mainly at distributions
to set their preferred default behavior in their kernels.

To allow users to override the distro's policy, introduce the
'aarch32_emulation' parameter which allows overriding
CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED state at boot time.

Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
 arch/arm64/Kconfig                              | 9 +++++++++
 arch/arm64/kernel/entry-common.c                | 8 +++++++-
 3 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 0a1731a0f0ef..a41c5e6f5d2e 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1,3 +1,10 @@
+	aarch32_emulation=	[ARM64]
+			Format: <bool>
+			When true, allows loading 32-bit programs and executing
+			32-bit syscalls, essentially overriding
+			AARCH32_EMULATION_DEFAULT_DISABLED at boot time. when false,
+			unconditionally disables AARCH32 emulation.
+
 	acpi=		[HW,ACPI,X86,ARM64,RISCV64]
 			Advanced Configuration and Power Interface
 			Format: { force | on | off | strict | noirq | rsdt |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b10515c0200b..66c4cb273550 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1725,6 +1725,15 @@ config SETEND_EMULATION
 	  If unsure, say Y
 endif # ARMV8_DEPRECATED
 
+config AARCH32_EMULATION_DEFAULT_DISABLED
+	bool "Aarch32 emulation disabled by default"
+	default n
+	depends on COMPAT
+	help
+	  Make Aarch32 emulation disabled by default. This prevents loading 32-bit
+	  processes and access to 32-bit syscalls.
+
+	  If unsure, leave it to its default value.
 endif # COMPAT
 
 menu "ARMv8.1 architectural features"
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 32761760d9dd..07f2b4e632b8 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -897,7 +897,13 @@ asmlinkage void noinstr el0t_32_error_handler(struct pt_regs *regs)
 		__el0_error_handler_common(regs);
 }
 
-bool __aarch32_enabled __ro_after_init = true;
+bool __aarch32_enabled __ro_after_init = !IS_ENABLED(CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED);
+
+static int aarch32_emulation_override_cmdline(char *arg)
+{
+	return kstrtobool(arg, &__aarch32_enabled);
+}
+early_param("aarch32_emulation", aarch32_emulation_override_cmdline);
 #else /* CONFIG_COMPAT */
 UNHANDLED(el0t, 32, sync)
 UNHANDLED(el0t, 32, irq)
-- 
2.35.3


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 4/4] arm64: Make Aarch32 emulation boot time configurable

[Mark Rutland]

On Wed, Oct 18, 2023 at 01:13:22PM +0200, Andrea della Porta wrote:
> Distributions would like to reduce their attack surface as much as
> possible but at the same time they'd want to retain flexibility to
> cater to a variety of legacy software. This stems from the conjecture
> that compat layer is likely rarely tested and could have latent
> security bugs. Ideally distributions will set their default policy
> and also give users the ability to override it as appropriate.
> 
> To enable this use case, introduce CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED
> compile time option, which controls whether 32bit processes/syscalls
> should be allowed or not. This option is aimed mainly at distributions
> to set their preferred default behavior in their kernels.
> 
> To allow users to override the distro's policy, introduce the
> 'aarch32_emulation' parameter which allows overriding
> CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED state at boot time.
> 
> Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
>  arch/arm64/Kconfig                              | 9 +++++++++
>  arch/arm64/kernel/entry-common.c                | 8 +++++++-
>  3 files changed, 23 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 0a1731a0f0ef..a41c5e6f5d2e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -1,3 +1,10 @@
> +	aarch32_emulation=	[ARM64]
> +			Format: <bool>
> +			When true, allows loading 32-bit programs and executing
> +			32-bit syscalls, essentially overriding
> +			AARCH32_EMULATION_DEFAULT_DISABLED at boot time. when false,
> +			unconditionally disables AARCH32 emulation.

Can we please drop the 'emulation' part of the name? We don't use that
terminology on arm64 for regular execution of compat tasks, and only use that
to refer to true emulation of deprecated instructions.

We already have the 'allow_mismatched_32bit_el0' option; can we please us a
name that aligns with that? e.g. 'allow_32bit_el0=false' to disable support.

Mark.

> +
>  	acpi=		[HW,ACPI,X86,ARM64,RISCV64]
>  			Advanced Configuration and Power Interface
>  			Format: { force | on | off | strict | noirq | rsdt |
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index b10515c0200b..66c4cb273550 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1725,6 +1725,15 @@ config SETEND_EMULATION
>  	  If unsure, say Y
>  endif # ARMV8_DEPRECATED
>  
> +config AARCH32_EMULATION_DEFAULT_DISABLED
> +	bool "Aarch32 emulation disabled by default"
> +	default n
> +	depends on COMPAT
> +	help
> +	  Make Aarch32 emulation disabled by default. This prevents loading 32-bit
> +	  processes and access to 32-bit syscalls.
> +
> +	  If unsure, leave it to its default value.
>  endif # COMPAT
>  
>  menu "ARMv8.1 architectural features"
> diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
> index 32761760d9dd..07f2b4e632b8 100644
> --- a/arch/arm64/kernel/entry-common.c
> +++ b/arch/arm64/kernel/entry-common.c
> @@ -897,7 +897,13 @@ asmlinkage void noinstr el0t_32_error_handler(struct pt_regs *regs)
>  		__el0_error_handler_common(regs);
>  }
>  
> -bool __aarch32_enabled __ro_after_init = true;
> +bool __aarch32_enabled __ro_after_init = !IS_ENABLED(CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED);
> +
> +static int aarch32_emulation_override_cmdline(char *arg)
> +{
> +	return kstrtobool(arg, &__aarch32_enabled);
> +}
> +early_param("aarch32_emulation", aarch32_emulation_override_cmdline);
>  #else /* CONFIG_COMPAT */
>  UNHANDLED(el0t, 32, sync)
>  UNHANDLED(el0t, 32, irq)
> -- 
> 2.35.3
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 4/4] arm64: Make Aarch32 emulation boot time configurable

[Andrea della Porta]

On 14:02 Wed 18 Oct     , Mark Rutland wrote:
> On Wed, Oct 18, 2023 at 01:13:22PM +0200, Andrea della Porta wrote:
> > Distributions would like to reduce their attack surface as much as
> > possible but at the same time they'd want to retain flexibility to
> > cater to a variety of legacy software. This stems from the conjecture
> > that compat layer is likely rarely tested and could have latent
> > security bugs. Ideally distributions will set their default policy
> > and also give users the ability to override it as appropriate.
> > 
> > To enable this use case, introduce CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED
> > compile time option, which controls whether 32bit processes/syscalls
> > should be allowed or not. This option is aimed mainly at distributions
> > to set their preferred default behavior in their kernels.
> > 
> > To allow users to override the distro's policy, introduce the
> > 'aarch32_emulation' parameter which allows overriding
> > CONFIG_AARCH32_EMULATION_DEFAULT_DISABLED state at boot time.
> > 
> > Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
> > ---
> >  Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
> >  arch/arm64/Kconfig                              | 9 +++++++++
> >  arch/arm64/kernel/entry-common.c                | 8 +++++++-
> >  3 files changed, 23 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 0a1731a0f0ef..a41c5e6f5d2e 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -1,3 +1,10 @@
> > +	aarch32_emulation=	[ARM64]
> > +			Format: <bool>
> > +			When true, allows loading 32-bit programs and executing
> > +			32-bit syscalls, essentially overriding
> > +			AARCH32_EMULATION_DEFAULT_DISABLED at boot time. when false,
> > +			unconditionally disables AARCH32 emulation.
> 
> Can we please drop the 'emulation' part of the name? We don't use that
> terminology on arm64 for regular execution of compat tasks, and only use that
> to refer to true emulation of deprecated instructions.
> 
> We already have the 'allow_mismatched_32bit_el0' option; can we please us a
> name that aligns with that? e.g. 'allow_32bit_el0=false' to disable support.
> 

Sure, 'allow_mismatched_32bit_el0' will do. I'll prepare a patch accordingly.

Andrea

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Will Deacon]

Hi,

On Wed, Oct 18, 2023 at 01:13:18PM +0200, Andrea della Porta wrote:
> Aarch32 compatibility mode is enabled at compile time through
> CONFIG_COMPAT Kconfig option. This patchset lets 32-bit support
> (for both processes and syscalls) be enabled at boot time using
> a kernel parameter. Also, it provides a mean for distributions 
> to set their own default without sacrificing compatibility support,
> that is users can override default behaviour through the kernel
> parameter.

I proposed something similar in the past:

https://lkml.kernel.org/linux-fsdevel/20210916131816.8841-1-will@kernel.org/

bu the conclusion there (see the reply from Kees) was that it was better
to either use existing seccomp mechanisms or add something to control
which binfmts can be loaded.

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Arnd Bergmann]

On Wed, Oct 18, 2023, at 14:27, Will Deacon wrote:
> Hi,
>
> On Wed, Oct 18, 2023 at 01:13:18PM +0200, Andrea della Porta wrote:
>> Aarch32 compatibility mode is enabled at compile time through
>> CONFIG_COMPAT Kconfig option. This patchset lets 32-bit support
>> (for both processes and syscalls) be enabled at boot time using
>> a kernel parameter. Also, it provides a mean for distributions 
>> to set their own default without sacrificing compatibility support,
>> that is users can override default behaviour through the kernel
>> parameter.
>
> I proposed something similar in the past:
>
> https://lkml.kernel.org/linux-fsdevel/20210916131816.8841-1-will@kernel.org/
>
> bu the conclusion there (see the reply from Kees) was that it was better
> to either use existing seccomp mechanisms or add something to control
> which binfmts can be loaded.

Right, I was going to reply along the same lines here: x86 is
a bit of a special case that needs this, but I believe all the
other architectures already guard the compat syscall execution
on test_thread_flag(TIF_32BIT) that is only set by the compat
binfmt loader.

Doing the reverse is something that has however come up in the
past several times and that could be interesting: In order to
run userspace emulation (qemu-user, fex, ...) we may want to
allow calling syscalls and ioctls for foreign ABIs in a native
task, and at that point having a mechanism to control this
capability globally or per task would be useful as well.

The compat mode (arm32 on arm64) is the easiest case here, but the
same thing could be done for emulating the very subtle architecture
differences (x86-64 on arm64, arm64 on x86_64, arm32 on x86-compat,
or any of the above on riscv or  loongarch).

     Arnd

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Andrea della Porta]

On 14:44 Wed 18 Oct     , Arnd Bergmann wrote:
> On Wed, Oct 18, 2023, at 14:27, Will Deacon wrote:
> > Hi,
> >
> > On Wed, Oct 18, 2023 at 01:13:18PM +0200, Andrea della Porta wrote:
> >> Aarch32 compatibility mode is enabled at compile time through
> >> CONFIG_COMPAT Kconfig option. This patchset lets 32-bit support
> >> (for both processes and syscalls) be enabled at boot time using
> >> a kernel parameter. Also, it provides a mean for distributions 
> >> to set their own default without sacrificing compatibility support,
> >> that is users can override default behaviour through the kernel
> >> parameter.
> >
> > I proposed something similar in the past:
> >
> > https://lkml.kernel.org/linux-fsdevel/20210916131816.8841-1-will@kernel.org/
> >
> > bu the conclusion there (see the reply from Kees) was that it was better
> > to either use existing seccomp mechanisms or add something to control
> > which binfmts can be loaded.
> 
> Right, I was going to reply along the same lines here: x86 is
> a bit of a special case that needs this, but I believe all the
> other architectures already guard the compat syscall execution
> on test_thread_flag(TIF_32BIT) that is only set by the compat
> binfmt loader.

Are you referring to the fact that x86 can switch at will between 32- and 64-
bit code?

Regarding the TIF_32BIT flag, thanks for the head-up. I still believe though
that this mechanism can somehow break down in the future, since prohibiting
32 bit executable loading *and* blocking 32 bit compat syscall are two
separate path of execution, held together by the architecture prohibiting
to switch to A32 instructions by design. Breaking the first rule and embedding 
wisely crafted A32 instruction in an executable is easy, while the difficult
part is finding some 'reentrancy' to be able to do the execution state switch,
as pinted out in https://lore.kernel.org/lkml/ZTD0DAes-J-YQ2eu@apocalypse/.
I agree it's highly speculative and not something to be concerned right
now, it's just a head up, should the need arise in the future.

> Doing the reverse is something that has however come up in the
> past several times and that could be interesting: In order to
> run userspace emulation (qemu-user, fex, ...) we may want to
> allow calling syscalls and ioctls for foreign ABIs in a native
> task, and at that point having a mechanism to control this
> capability globally or per task would be useful as well.
> 
> The compat mode (arm32 on arm64) is the easiest case here, but the
> same thing could be done for emulating the very subtle architecture
> differences (x86-64 on arm64, arm64 on x86_64, arm32 on x86-compat,
> or any of the above on riscv or  loongarch).
> 
>      Arnd

Really interesting, Since it's more related to emulation needs (my patch
has another focus due to the fact that A64 can execute A32 natively),
I'll take a look at this separately.

Andrea 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Arnd Bergmann]

On Thu, Oct 19, 2023, at 12:52, Andrea della Porta wrote:
> On 14:44 Wed 18 Oct     , Arnd Bergmann wrote:
>> On Wed, Oct 18, 2023, at 14:27, Will Deacon wrote:
>> 
>> Right, I was going to reply along the same lines here: x86 is
>> a bit of a special case that needs this, but I believe all the
>> other architectures already guard the compat syscall execution
>> on test_thread_flag(TIF_32BIT) that is only set by the compat
>> binfmt loader.
>
> Are you referring to the fact that x86 can switch at will between 32- and 64-
> bit code?

No.

> Regarding the TIF_32BIT flag, thanks for the head-up. I still believe though
> that this mechanism can somehow break down in the future, since prohibiting
> 32 bit executable loading *and* blocking 32 bit compat syscall are two
> separate path of execution, held together by the architecture prohibiting
> to switch to A32 instructions by design. Breaking the first rule and embedding 
> wisely crafted A32 instruction in an executable is easy, while the difficult
> part is finding some 'reentrancy' to be able to do the execution state switch,
> as pinted out in https://lore.kernel.org/lkml/ZTD0DAes-J-YQ2eu@apocalypse/.
> I agree it's highly speculative and not something to be concerned right
> now, it's just a head up, should the need arise in the future.

There are (at least) five separate aspects to compat mode that are easy
to mix up:

1. Instruction decoding -- switching between the modes supported by the
   CPU (A64/A32/T32)
2. Word size -- what happens to the upper 32 bits of a register in
   an arithmetic operation
3. Personality -- Which architecture string gets returned by the
   uname syscall (aarch64 vs armv8) as well as the format of
   /proc/cpuinfo
4. system call entry points -- how a process calls into native or
   compat syscalls, or possibly foreign OS emulation
5. Binary format -- elf32 vs elf64 executables

On most architectures with compat mode, 4. and 5. are fundamentally
tied together today: a compat task can only call compat syscalls
and a native task can only call native syscalls. x86 is the exception
here, as it uses different instructions (int80, syscall, sysenter)
and picks the syscall table based on that instruction.

I think 1. and 2. are also always tied to 5 on arm, but this is
not necessarily true for other architectures. 3. used to be tied
to 5 on some architectures in the past, but should be independent
now.

>> Doing the reverse is something that has however come up in the
>> past several times and that could be interesting: In order to
>> run userspace emulation (qemu-user, fex, ...) we may want to
>> allow calling syscalls and ioctls for foreign ABIs in a native
>> task, and at that point having a mechanism to control this
>> capability globally or per task would be useful as well.
>> 
>> The compat mode (arm32 on arm64) is the easiest case here, but the
>> same thing could be done for emulating the very subtle architecture
>> differences (x86-64 on arm64, arm64 on x86_64, arm32 on x86-compat,
>> or any of the above on riscv or  loongarch).
>
> Really interesting, Since it's more related to emulation needs (my patch
> has another focus due to the fact that A64 can execute A32 natively),
> I'll take a look at this separately.

A64 mode (unlike some other architectures, notably mips64) cannot
execute A32 or T32 instructions without a mode switch, the three are
entirely incompatible on the binary level.

Many ARMv8-CPUs support both Aarch64 mode and Aarch32 (A32/T32),
but a lot of the newer ones (e.g. Apple M1/M2, Cortex-R82 or
Cortex-A715) only do Aarch64 and need user-space emulation to run
32-bit binaries.

    Arnd

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Andrea della Porta]

On 13:27 Wed 18 Oct     , Will Deacon wrote:
> Hi,
> 
> On Wed, Oct 18, 2023 at 01:13:18PM +0200, Andrea della Porta wrote:
> > Aarch32 compatibility mode is enabled at compile time through
> > CONFIG_COMPAT Kconfig option. This patchset lets 32-bit support
> > (for both processes and syscalls) be enabled at boot time using
> > a kernel parameter. Also, it provides a mean for distributions 
> > to set their own default without sacrificing compatibility support,
> > that is users can override default behaviour through the kernel
> > parameter.
> 
> I proposed something similar in the past:
> 
> https://lkml.kernel.org/linux-fsdevel/20210916131816.8841-1-will@kernel.org/
> 
> bu the conclusion there (see the reply from Kees) was that it was better
> to either use existing seccomp mechanisms or add something to control
> which binfmts can be loaded.
> 
> Will

I see. Seccomp sounds like a really good idea, since just blocking the compat
binfmt would not avoid the call to 32-bit syscalls per se: it's true that
ARM64 enforce the transition from A64 to A32 only on exception return and
PSTATE.nRW flag can change only from EL1, maybe though some exploitation
may arise in the future to do just that (I'm not aware of any or come up
with a proof off the top of my head, but I can't exclude it either). So,
assuming by absurd a switch to A32 is feasible, the further step of embedding
A32 instruction in a A64 ELF executable is a breeze. Hence blocking the 
syscall (and not only the binfmt loading) could prove necessary. I know all
of this is higly speculative right now, maybe it's worth thinking nonetheless.

Andrea

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Mark Rutland]

On Wed, Oct 18, 2023 at 01:13:18PM +0200, Andrea della Porta wrote:
> Aarch32 compatibility mode is enabled at compile time through
> CONFIG_COMPAT Kconfig option. This patchset lets 32-bit support
> (for both processes and syscalls) be enabled at boot time using
> a kernel parameter. Also, it provides a mean for distributions 
> to set their own default without sacrificing compatibility support,
> that is users can override default behaviour through the kernel
> parameter.

Can you elaborate on *why* people want such a policy?

> *** Notes about syscall management ***
> VBAR_EL1 register, which holds the exception table address,
> is setup very early in the boot process, before parse_early_param().
> This means that it's not possible to access boot parameter before
> setting the register. Also, setting the aforementioned register
> for secondary cpus is done later in the boot flow.
> Several ways to work around this has been considered, among which:
> 
> * resetting VBAR_EL1 to point to one of two vector tables (the
>   former with 32-bit exceptions handler enabled and the latter
>   pointing to unhandled stub, just as if CONFIG_COMPAT is enabled)
>   depending on the proposed boot parameter. This has the disadvantage
>   to produce a somewhat messy patchset involving several lines,
>   has higher cognitive load since there are at least three places
>   where the register is getting changed (not near to each other),
>   and have implications on other code segments (namely kpti, kvm
>   and vdso), requiring special care.
> 
> * patching the vector table contents once the early param is available.
>   This has most of the implications of the previous option
>   (except maybe not impacting other code segments), plus it sounds
>   a little 'hackish'.
> 
> The chosen approach involves conditional executing 32-bit syscalls
> depending on the parameter value.

Why does the compat syscall path need to do anything?

On arm64 it's not possible to issue compat syscalls from a native 64-bit task.
If you prevent the loading of AArch32 binaries, none of the compat syscalls
will be reachable at all.

That's the proper way to implement this, and we already have logic for that as
part of the mismatched AArch32 support.

> This of course results in a little performance loss, but has the following
> advantages:

A performance loss for what relative to what?

How much of a performance loss?

Mark.

> * all the cons from previously explained alternatives are solved
> * users of 32-bit apps on 64-bit kernel are already suffering from
>   performance losses due to 32-bit apps not fully leveraging the 64-bit
>   processor, so they are already aware of this
> * users of 32-bit apps on 64-bit kernel are believed
>   to be a minority and most of the time there are sources available
>   to be recompiled for 64-bit as a workaround for better performance
> 
> It worth mentioning that users of 64-bit apps are, of course,
> unaffected.
> 
> Based on the work from Nikolay Borisov, see:
> Link: https://lkml.org/lkml/2023/6/23/387
> 
> Andrea della Porta (4):
>   arm64: Introduce aarch32_enabled()
>   arm64/process: Make loading of 32bit processes depend on
>     aarch32_enabled()
>   arm64/entry-common: Make Aarch32 syscalls' availability depend on
>     aarch32_enabled()
>   arm64: Make Aarch32 emulation boot time configurable
> 
>  .../admin-guide/kernel-parameters.txt         |  7 ++++
>  arch/arm64/Kconfig                            |  9 +++++
>  arch/arm64/include/asm/compat.h               | 12 +++++++
>  arch/arm64/kernel/entry-common.c              | 33 +++++++++++++++++--
>  arch/arm64/kernel/process.c                   |  2 +-
>  5 files changed, 59 insertions(+), 4 deletions(-)
> 
> -- 
> 2.35.3
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/4] arm64: Make Aarch32 compatibility enablement optional at boot

[Andrea della Porta]

On 13:52 Wed 18 Oct     , Mark Rutland wrote:
> On Wed, Oct 18, 2023 at 01:13:18PM +0200, Andrea della Porta wrote:
> > Aarch32 compatibility mode is enabled at compile time through
> > CONFIG_COMPAT Kconfig option. This patchset lets 32-bit support
> > (for both processes and syscalls) be enabled at boot time using
> > a kernel parameter. Also, it provides a mean for distributions 
> > to set their own default without sacrificing compatibility support,
> > that is users can override default behaviour through the kernel
> > parameter.
> 
> Can you elaborate on *why* people want such a policy?
> 

Formerly, the reason was to reduce kernel attack surface by excluding
compat syscall, wherever applicable. Much less important but still a point,
I would also say this could be a good chance to get rid of somewhat old
and stale 32-bit libraries and programs, but this is of course debatable.

> > *** Notes about syscall management ***
> > VBAR_EL1 register, which holds the exception table address,
> > is setup very early in the boot process, before parse_early_param().
> > This means that it's not possible to access boot parameter before
> > setting the register. Also, setting the aforementioned register
> > for secondary cpus is done later in the boot flow.
> > Several ways to work around this has been considered, among which:
> > 
> > * resetting VBAR_EL1 to point to one of two vector tables (the
> >   former with 32-bit exceptions handler enabled and the latter
> >   pointing to unhandled stub, just as if CONFIG_COMPAT is enabled)
> >   depending on the proposed boot parameter. This has the disadvantage
> >   to produce a somewhat messy patchset involving several lines,
> >   has higher cognitive load since there are at least three places
> >   where the register is getting changed (not near to each other),
> >   and have implications on other code segments (namely kpti, kvm
> >   and vdso), requiring special care.
> > 
> > * patching the vector table contents once the early param is available.
> >   This has most of the implications of the previous option
> >   (except maybe not impacting other code segments), plus it sounds
> >   a little 'hackish'.
> > 
> > The chosen approach involves conditional executing 32-bit syscalls
> > depending on the parameter value.
> 
> Why does the compat syscall path need to do anything?

I probably didn't catch your point here, compat syscall does not need to do
anything and they do not (just like they works right now with CONFIG_COMPAT
alone), except for the conditional instruction that excludes them at runtime.
Of course this conditional *is* doing something and somewhat redundant if
compat is disabled, but in this scenario I think it's unavoidable.

> 
> On arm64 it's not possible to issue compat syscalls from a native 64-bit task.
> If you prevent the loading of AArch32 binaries, none of the compat syscalls
> will be reachable at all.
> 
> That's the proper way to implement this, and we already have logic for that as
> part of the mismatched AArch32 support.
> 
> > This of course results in a little performance loss, but has the following
> > advantages:
> 
> A performance loss for what relative to what?

of a compat syscall as it is now enabling CONFIG_COMPAT vs the patched
syscall handlers that need a further conditional instruction to check
whether comapt is enabled or not.

> 
> How much of a performance loss?

I did not take measurement yet since it was just a qualitative consideration
more than a quantitative one, also considering that chances are that it would
affect just very little population. The conditional instruction time taken
to execute is reasonably near to negligible if compared to any syscall execution.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel