[PATCH v2] riscv: smp: Align secondary_start_sbi to 4 bytes

[PATCH v2] riscv: smp: Align secondary_start_sbi to 4 bytes

[cp0613]

From: Chen Pei <cp0613@linux.alibaba.com>

During SMP boot, the secondary_start_sbi address is passed to the
slave core via sbi_hsm_hart_start. In OpenSBI, this address is
written to STVEC in sbi_hart_switch_mode.

According to the privileged specification, the BASE field of STVEC
must always be aligned on a 4-byte boundary. However, System.map
reveals that secondary_start_sbi is not a 4-byte aligned address.

For example, the address of secondary_start_sbi is
0xffffffff80001066, and the disassembly is as follows:

Dump of assembler code from 0xffffffff80001052 to 0xffffffff8000107a:
   0xffffffff80001052 <_start+4178>:    c.nop   -11
   0xffffffff80001054 <_start+4180>:    auipc   gp,0x1a1f
   0xffffffff80001058 <_start+4184>:    addi    gp,gp,84
   0xffffffff8000105c <_start+4188>:    csrw    satp,a2
   0xffffffff80001060 <_start+4192>:    sfence.vma
   0xffffffff80001064 <_start+4196>:    ret
   0xffffffff80001066 <_start+4198>:    csrw    sie,zero
   0xffffffff8000106a <_start+4202>:    csrw    sip,zero
   0xffffffff8000106e <_start+4206>:    li      t0,2
   0xffffffff80001070 <_start+4208>:    csrw    scounteren,t0
   0xffffffff80001074 <_start+4212>:    auipc   gp,0x1a1f
   0xffffffff80001078 <_start+4216>:    addi    gp,gp,52

When writing to STVEC at address 0xffffffff80001066, the actual
write address is 0xffffffff80001064, corresponding to the address
of the previous ret instruction. This is unexpected, and if an
interrupt occurs at this point, it will cause unpredictable results.

However, secondary_start_sbi immediately masks all interrupts and
updates STVEC, so no problems occurred.

In summary, it is more reasonable to make secondary_start_sbi
satisfy 4-byte alignment.

Changes in v2:
 - Place `.align 2` inside `#ifdef CONFIG_SMP`, above the tag.
 - Add two Reviewed-by tags.
 - Based on Linux 7.0.

Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
---
 arch/riscv/kernel/head.S | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index 9c99c5ad6fe8..9f33be6260e1 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -127,6 +127,7 @@ relocate_enable_mmu:
 #endif /* CONFIG_MMU */
 #ifdef CONFIG_SMP
 	.global secondary_start_sbi
+	.align 2
 secondary_start_sbi:
 	/* Mask all interrupts */
 	csrw CSR_IE, zero
-- 
2.50.1

Re: [PATCH v2] riscv: smp: Align secondary_start_sbi to 4 bytes

[Michael Ellerman]

On 13/4/2026 23:20, cp0613@linux.alibaba.com wrote:
> From: Chen Pei <cp0613@linux.alibaba.com>
> 
> During SMP boot, the secondary_start_sbi address is passed to the
> slave core via sbi_hsm_hart_start. In OpenSBI, this address is
> written to STVEC in sbi_hart_switch_mode.

...

> diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
> index 9c99c5ad6fe8..9f33be6260e1 100644
> --- a/arch/riscv/kernel/head.S
> +++ b/arch/riscv/kernel/head.S
> @@ -127,6 +127,7 @@ relocate_enable_mmu:
>   #endif /* CONFIG_MMU */
>   #ifdef CONFIG_SMP
>   	.global secondary_start_sbi
> +	.align 2
>   secondary_start_sbi:
>   	/* Mask all interrupts */
>   	csrw CSR_IE, zero


Minor nit, but IMHO .balign is preferable in new code. It always byte 
aligns, whereas .align has different meanings across architectures, and 
on some arches (including riscv) requires the reader to do the math to 
convert to a byte alignment.

These days there's also the SYM_CODE_START etc. macros defined via 
linkage.h, which use __ALIGN from arch/riscv/include/asm/linkage.h, 
which is already .balign 4.

cheers

Re: [PATCH v2] riscv: smp: Align secondary_start_sbi to 4 bytes

[Vivian Wang]

[ +cc opensbi, Anup: What was the deal with setting stvec in sbi_hart_switch_mode? ]

On 4/13/26 21:20, cp0613@linux.alibaba.com wrote:

> From: Chen Pei <cp0613@linux.alibaba.com>
>
> During SMP boot, the secondary_start_sbi address is passed to the
> slave core via sbi_hsm_hart_start. In OpenSBI, this address is
> written to STVEC in sbi_hart_switch_mode.

This is certainly odd (OpenSBI git blames to initial git commit. No idea
why this was needed.) but technically permitted by the SBI HSM spec (see
below).

* Anup and other opensbi folks: Do you know/remember why it was there?
It's not really a bug, but some historical context would be appreciated.

> According to the privileged specification, the BASE field of STVEC
> must always be aligned on a 4-byte boundary. However, System.map
> reveals that secondary_start_sbi is not a 4-byte aligned address.
>
> For example, the address of secondary_start_sbi is
> 0xffffffff80001066, and the disassembly is as follows:
>
> Dump of assembler code from 0xffffffff80001052 to 0xffffffff8000107a:
>    0xffffffff80001052 <_start+4178>:    c.nop   -11
>    0xffffffff80001054 <_start+4180>:    auipc   gp,0x1a1f
>    0xffffffff80001058 <_start+4184>:    addi    gp,gp,84
>    0xffffffff8000105c <_start+4188>:    csrw    satp,a2
>    0xffffffff80001060 <_start+4192>:    sfence.vma
>    0xffffffff80001064 <_start+4196>:    ret
>    0xffffffff80001066 <_start+4198>:    csrw    sie,zero
>    0xffffffff8000106a <_start+4202>:    csrw    sip,zero
>    0xffffffff8000106e <_start+4206>:    li      t0,2
>    0xffffffff80001070 <_start+4208>:    csrw    scounteren,t0
>    0xffffffff80001074 <_start+4212>:    auipc   gp,0x1a1f
>    0xffffffff80001078 <_start+4216>:    addi    gp,gp,52
>
> When writing to STVEC at address 0xffffffff80001066, the actual
> write address is 0xffffffff80001064, corresponding to the address
> of the previous ret instruction.
Also technically permitted.
> This is unexpected, and if an
> interrupt occurs at this point, it will cause unpredictable results.
>
> However, secondary_start_sbi immediately masks all interrupts and
> updates STVEC, so no problems occurred.

An interrupt cannot trap in between secondary_start_sbi starting and the
interrupts getting masked individually. The only arch state guaranteed
by HSM is [1]:

|Register Name | Register Value
|satp          | 0
|sstatus.SIE   | 0
|a0            | hartid
|a1            | `opaque` parameter
|All other registers remain in an undefined state.

Of which sstatus.SIE = 0 means that interrupts are masked in Supervisor
mode.
> In summary, it is more reasonable to make secondary_start_sbi
> satisfy 4-byte alignment.

Thus, it is unnecessary.

If this fixes a bug where an interrupt can trap into the wrong stvec for
you, then your firmware is broken.

Michael's suggestion about SYM_CODE_START still stands, but if you do
that, that's just a really minor refactoring, and in any case the
current commit message cannot stand.

Vivian "dramforever" Wang

[1]:
https://github.com/riscv-non-isa/riscv-sbi-doc/blob/v3.0/src/ext-hsm.adoc#function-hart-start-fid-0

Re: [PATCH v2] riscv: smp: Align secondary_start_sbi to 4 bytes

[Chen Pei]

On Wed, 15 Apr 2026 12:06:28 +1000, mpe@kernel.org wrote:

> > diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
> > index 9c99c5ad6fe8..9f33be6260e1 100644
> > --- a/arch/riscv/kernel/head.S
> > +++ b/arch/riscv/kernel/head.S
> > @@ -127,6 +127,7 @@ relocate_enable_mmu:
> >   #endif /* CONFIG_MMU */
> >   #ifdef CONFIG_SMP
> >    .global secondary_start_sbi
> > + .align 2
> >   secondary_start_sbi:
> >    /* Mask all interrupts */
> >    csrw CSR_IE, zero
> 
> 
> Minor nit, but IMHO .balign is preferable in new code. It always byte 
> aligns, whereas .align has different meanings across architectures, and 
> on some arches (including riscv) requires the reader to do the math to 
> convert to a byte alignment.
> 
> These days there's also the SYM_CODE_START etc. macros defined via 
> linkage.h, which use __ALIGN from arch/riscv/include/asm/linkage.h, 
> which is already .balign 4.

Hi Michael,

Thank you for your review and reminder. If this patch is ultimately
approved (because the behavior of OpenSBI still needs clarification
based on the new reviewer's comments), I will update it using .balign.

Thanks,
Pei

Re: [PATCH v2] riscv: smp: Align secondary_start_sbi to 4 bytes

[Chen Pei]

On Wed, 15 Apr 2026 11:39:22 +0800, wangruikang@iscas.ac.cn wrote:

> > During SMP boot, the secondary_start_sbi address is passed to the
> > slave core via sbi_hsm_hart_start. In OpenSBI, this address is
> > written to STVEC in sbi_hart_switch_mode.
> 
> This is certainly odd (OpenSBI git blames to initial git commit. No idea
> why this was needed.) but technically permitted by the SBI HSM spec (see
> below).
> 
> * Anup and other opensbi folks: Do you know/remember why it was there?
> It's not really a bug, but some historical context would be appreciated.

Yes, looking forward to responses from Anup and other opensbi folks.

> > According to the privileged specification, the BASE field of STVEC
> > must always be aligned on a 4-byte boundary. However, System.map
> > reveals that secondary_start_sbi is not a 4-byte aligned address.
> >
> > For example, the address of secondary_start_sbi is
> > 0xffffffff80001066, and the disassembly is as follows:
> >
> > Dump of assembler code from 0xffffffff80001052 to 0xffffffff8000107a:
> >    0xffffffff80001052 <_start+4178>:    c.nop   -11
> >    0xffffffff80001054 <_start+4180>:    auipc   gp,0x1a1f
> >    0xffffffff80001058 <_start+4184>:    addi    gp,gp,84
> >    0xffffffff8000105c <_start+4188>:    csrw    satp,a2
> >    0xffffffff80001060 <_start+4192>:    sfence.vma
> >    0xffffffff80001064 <_start+4196>:    ret
> >    0xffffffff80001066 <_start+4198>:    csrw    sie,zero
> >    0xffffffff8000106a <_start+4202>:    csrw    sip,zero
> >    0xffffffff8000106e <_start+4206>:    li      t0,2
> >    0xffffffff80001070 <_start+4208>:    csrw    scounteren,t0
> >    0xffffffff80001074 <_start+4212>:    auipc   gp,0x1a1f
> >    0xffffffff80001078 <_start+4216>:    addi    gp,gp,52
> >
> > When writing to STVEC at address 0xffffffff80001066, the actual
> > write address is 0xffffffff80001064, corresponding to the address
> > of the previous ret instruction.
> Also technically permitted.

Yes, but this is unintended behavior, and we should avoid it.

> > This is unexpected, and if an
> > interrupt occurs at this point, it will cause unpredictable results.
> >
> > However, secondary_start_sbi immediately masks all interrupts and
> > updates STVEC, so no problems occurred.
> 
> An interrupt cannot trap in between secondary_start_sbi starting and the
> interrupts getting masked individually. The only arch state guaranteed
> by HSM is [1]:
> 
> |Register Name | Register Value
> |satp          | 0
> |sstatus.SIE   | 0
> |a0            | hartid
> |a1            | `opaque` parameter
> |All other registers remain in an undefined state.
> 
> Of which sstatus.SIE = 0 means that interrupts are masked in Supervisor
> mode.
> > In summary, it is more reasonable to make secondary_start_sbi
> > satisfy 4-byte alignment.
> 
> Thus, it is unnecessary.
> 
> If this fixes a bug where an interrupt can trap into the wrong stvec for
> you, then your firmware is broken.

This isn't about fixing a bug (as mentioned earlier, it's highly unlikely
to happen), but rather the STVEC check mechanism added to QEMU discovered
this issue. Regarding the problem itself, I would appreciate any clarification
or modifications from OpenSBI, but currently this modification is the simplest
and most feasible.

> Michael's suggestion about SYM_CODE_START still stands, but if you do
> that, that's just a really minor refactoring, and in any case the
> current commit message cannot stand.
> 
> Vivian "dramforever" Wang
> 
> [1]:
> https://github.com/riscv-non-isa/riscv-sbi-doc/blob/v3.0/src/ext-hsm.adoc#function-hart-start-fid-0

Thanks,
Pei

Re: [PATCH v2] riscv: smp: Align secondary_start_sbi to 4 bytes

[Vivian Wang]

On 4/15/26 20:37, Chen Pei wrote:

[...]

>>> In summary, it is more reasonable to make secondary_start_sbi
>>> satisfy 4-byte alignment.
>> Thus, it is unnecessary.
>>
>> If this fixes a bug where an interrupt can trap into the wrong stvec for
>> you, then your firmware is broken.
> This isn't about fixing a bug (as mentioned earlier, it's highly unlikely
> to happen),

It is *impossible* given correct HW and FW, but okay...

> but rather the STVEC check mechanism added to QEMU discovered
> this issue. Regarding the problem itself, I would appreciate any clarification
> or modifications from OpenSBI, but currently this modification is the simplest
> and most feasible.

The stvec CSR is WARL. If the "STVEC check mechanism" is complaining
about usage of CSRs as specified, then the check is a false positive and
is not something fixable.

Vivian "dramforever" Wang