Re: [PATCH v4 19/27] s390: Introduce Start Arm Execution instruction

[Janosch Frank <frankja@linux.ibm.com>]

On 7/6/26 10:52, Steffen Eiden wrote:
> The Start Arm Execution (SAE) instruction is the centerpiece for
> executing arm64 (KVM) guests on s390. Its purpose is, similar to SIE, to
> enable accelerated execution of arm64 virtual machines. SAE expects the
> physical address of a control block as the only argument.
> 
> The host is responsible to save & restore
>   - GPRs 0-13
>   - access register 0-15
>   - breaking event register (BEAR)
>   - vector/floating point registers
> between SAE executions to guarantee host consistency.
> 
> GPRs and BEAR are save and restores in the asm functions. The other
> register are handled in within C code. Access registers are handled in a
> later patch and SVEs will be handled when they are introduced in a
> future series. Most arm64 registers are handled by a satellite block
> called save_area. Some registers, frequently used by hypervisors, are
> placed into the SAE control block itself.
> 
> Enlighten asm/kvm_host_types.h for the new header variant. The new
> header is chosen instead of asm/kvm_host_s390_types.h if KVM_S390_ARM64
> is defined.
> 

[...]

> +	u64	gpr[31];		/* 0x0300  */
> +	u64	_03f8;			/* 0x03f8  */
> +
> +	union {
> +		u64	icptd[8];		/* 0x0400 */
> +		/* validity-interception reason; icptr 0x01 */
> +#define SAE_VIR_UNKNOWN		0x00
> +#define SAE_VIR_UNSUPP_FORMAT	0x01
> +#define SAE_VIR_MSO_BOUNDS	0x02
> +#define SAE_VIR_MSLA		0x03
> +#define SAE_VIR_MGPAS		0x04
> +#define SAE_VIR_INVAL_SYSREG	0x05
> +#define SAE_VIR_HOST_CONTROL	0x06
> +#define SAE_VIR_SCA		0x07
> +#define SAE_VIR_MSO_ALIGN	0x08
> +#define SAE_VIR_HLC		0x09
> +#define SAE_VIR_IRPTC		0x0a

Will these ever be used for something?



> +		u16 vir;			/* 0x0400 */
> +		/* host access interception details; icptr 0x02 */
> +		struct {
> +			u64		esr_elz;	/* 0x0400 */
> +			u8		_0408[6];	/* 0x0408 */
> +			u16		pic;		/* 0x040e */
> +			union teid	teid;		/* 0x0410 */
> +			gva_t		far_elz;	/* 0x0418 */
> +			gva_t		vaddr;		/* 0x0420 */
> +			u64		suppl;		/* 0x0428 */
> +			u8		gltl;		/* 0x0430 */
> +			u8		_0431[7];	/* 0x0431 */
> +			u64		_0438;		/* 0x0438 */
> +		} hai;
> +		/* exception-interception details; icptr 0x03 */
> +		struct {
> +			u64	esr_elz;		/* 0x0400 */
> +			u64	_0408[2];		/* 0x0408 */
> +			gva_t	far_elz;		/* 0x0418 */
> +		} trap;
> +		/* timer-interception reason; icptr 0x04 */
> +#define SAE_IR_TIMER_ID_VIRT		BIT(6)
> +#define SAE_IR_TIMER_ID_PHYS		BIT(7)
> +		u8	tir;			/* 0x0400 */
> +	};
> +	u64	_0440[376];			/* 0x0440 */
> +} __packed __aligned(PAGE_SIZE);
> +static_assert(sizeof(struct kvm_sae_block) == PAGE_SIZE);
> +

[...]

> +#if IS_ENABLED(CONFIG_KVM_ARM64)
> +/*
> + * __sae64a calling convention:
> + * %r2 pointer to sae control block physical address
> + */
> +SYM_FUNC_START(__sae64a)
> +	stmg	%r6,%r14,__SF_GPRS(%r15)	# store kernel registers
> +	STBEAR	__SF_SAE_BEAR(%r15)		# save breaking event address register
> +	.insn	rre,0xb9a50000,%r2,0		# Start Arm Execution
> +# Let the next instruction be NOP to avoid triggering a machine check
> +# and handling it in a guest as result of the instruction execution.

@Christian:
I have the feeling things got lost in translation when this comment was 
written.
How would we trigger a MCHECK by having an instruction after SIE/SAE?
AFAIU this allows us to have a label which we can check in the mcheck 
handler to decide who triggered the mcheck.

Without the label the comment and NOP are useless as far as I can see.
So we need to decide if this still applies and then either remove it or 
add sae handling to mcck_int_handler and maybe other mcheck functions.

> +	nopr	7
> +	LBEAR	__SF_SAE_BEAR(%r15)		# restore breaking event address register
> +	lmg	%r6,%r14,__SF_GPRS(%r15)	# restore kernel registers
> +	xgr	%r0,%r0				# clear guest registers to
> +	xgr	%r1,%r1				# prevent speculative use
> +	xgr	%r3,%r3
> +	xgr	%r2,%r2
> +	xgr	%r4,%r4
> +	xgr	%r5,%r5
> +	BR_EX	%r14
> +SYM_FUNC_END(__sae64a)
> +EXPORT_SYMBOL(__sae64a)
> +#endif
> +
>   /*
>    * SVC interrupt handler routine. System calls are synchronous events and
>    * are entered with interrupts disabled.
> diff --git a/arch/s390/tools/opcodes.txt b/arch/s390/tools/opcodes.txt
> index def2659f6602..0e4773c94af0 100644
> --- a/arch/s390/tools/opcodes.txt
> +++ b/arch/s390/tools/opcodes.txt
> @@ -594,6 +594,9 @@ b9a0	clp	RRF_U0RR
>   b9a1	tpei	RRE_RR
>   b9a2	ptf	RRE_R0
>   b9a4	uvc	RRF_URR
> +b9a5	sae	RRE_R0
> +b9a6	lasrm	RRE_R0
> +b9a7	stiasrm	RRE_R0
>   b9aa	lptea	RRF_RURR2
>   b9ab	essa	RRF_U0RR
>   b9ac	irbm	RRE_RR