* [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support
@ 2025-05-19 15:50 Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 2/8] xen/arm: scmi-smc: update to be used under sci subsystem Oleksii Moisieiev
` (8 more replies)
0 siblings, 9 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
Inroducing V4 RFC patch series on top of the Xen version 4.20-rc2
which includes implementation of the SCI SCMI SMC multi-agent support.
Patch 1 "xen/arm: add generic SCI subsystem"
- rebased and refactored
- introduced DEVICE_ARM_SCI DT device class and used for SCI drivers probing
instead of custom,
linker sections based implementation.
- added SCI API for Dom0 DT handling, instead of manipulating with ARM arch
dom0 code directly.
- RFC changes in XEN_DOMCTL_assign_device OP processing
Patch 2 "xen/arm: scmi-smc: update to be used under sci subsystem"
- update driver introduced by commit 3e322bef8bc0 ("xen/arm: firmware: Add SCMI
over SMC calls
handling layer") be used under sci subsystem.
- no functional changes in general
Patch 3 "xen/arm: scmi-smc: passthrough SCMI SMC to guest domain
This is new change which allows passthrough SCMI SMC, single agent interface to
guest domain
cover use case "thin Dom0 with guest domain, which serves as Driver domain".
See patch commit message for full description.
Patch 4 - docs: arm: add docs for SCMI over SMC calls forwarding
driver
- add documentation section for Simple Arm SCMI over SMC/HVC calls
forwarding driver.
Patch 5 - xen/domctl: extend XEN_DOMCTL_assign_device to handle not
only iommu
- add chainged handling of assigned DT devices to support
access-controller functionality through SCI framework.
Change was done in two parts:
- update iommu_do_dt_domctl() to check for dt_device_is_protected()
and not fail if DT device is not protected by IOMMU
-add chained call to sci_do_domctl() to do_domctl()
Patch 6 - xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
- added "xen,scmi-secondary-agents" property in "chosen" to inform SCI SCMI
multi-agent driver
about available agents and their configuration. It defines <agent_id> to
<smc-id,scmi_shm> map.
This option is Xen specific as Xen is the only one entry in the system which
need to know
about SCMI multi-agent support and configuration.
- each guest using SCMI should be configured with SCMI agent_id, so SCMI
FW can implement Agent-specific permission policy.
-- dom0: dom0_scmi_agent_id=<agent_id> in Xen command line option
-- toolstack: arm_sci = "type=scmi_smc_multiagent,agent_id=<agent_id>"
-- dom0less: "xen,sci_type", "xen,sci_agent_id" properties in
"xen,domain" nodes.
- factored out SCMI generic definitions (re-usable)
- factored out SCMI shmem code (re-usable)
- the SCMI passthrough configuration for guest domains is similar to any other
HW passthrough cfg.
Patch 7 - docs: arm: add SCI SCMI SMC multi-agent driver docs
- add SCI SCMI SMC multi-agent driver documentation.
Patch 8 - docs: arm: proposal to add separate SCMI node for Xen agent
- proposal to add separate SCMI DT node for Xen management agent under
chosen of xen-config node, lile Hyperlaunch "xen,config".
This proposal introduces a new approach to the Xen multi-domain
configuration, where all Xen-specific configuration has been moved
under the "/chosen" node. This requires less Dom0 device tree
manipulation and isolates Xen configuration from domain configuration.
This approach provides the following device tree (DT) parameters:
- "xen,scmi-secondary-agents": A Xen-specific parameter under the
"/chosen" node, which describes the SCMI agent configuration for
the domains.
- the SCMI configuration for Xen (privileged agent) and the shared
memory configuration for all agents are provided under the "/chosen"
node and are used strictly by Xen for its initial configuration.
- the scmi_shm and SCMI configuration for Dom0 are placed in the
"/firmware/scmi" node so that they can be moved to Dom0 without
any changes.
This configuration allows the use of Xen-specific nodes to provide
information strictly needed by Xen while using the default SCMI
configuration for Dom0 and other domains. As a result, no additional
bindings need to be introduced to the device tree.
This simplifies the Xen SCMI multi-agent configuration and utilizes
generic device tree bindings for the domains.
Code can be found at:
https://github.com/oleksiimoisieiev/xen/tree/scmi_upstrv4
[1] RFC v2:
http://patchwork.kernel.org/project/xen-devel/cover/cover.1644341635.git.oleksii_moisieiev@epam.com/
[2] RFC v3:
https://patchwork.kernel.org/project/xen-devel/patch/20250311111618.1850927-1-grygorii_strashko@epam.com
SCMI spec:
https://developer.arm.com/documentation/den0056/e/?lang=en
SCMI bindings:
https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
Reference EL3 FW:
RPI5: https://github.com/xen-troops/arm-trusted-firmware/commits/rpi5_dev/
Renesas v4h:
https://github.com/GrygiriiS/arm-trusted-firmware/commits/rcar_gen4_v2.7_v4x-scmi_upd/
base-commit: dbe60f244c (Update Xen to 4.21, 2025-02-21)
Changes in v4:
- fix SPDX-License
- rename DEVICE_ARM_SCI DT device class to FIRMWARE_DEVICE
- move XEN_DOMCTL_assign_device code in separate patch
- Add documentation for SCI SCMI drivers
- xl.cfg doc
- fix comments from Stefano Stabellini
- fix toolstack code as sugested by Anthony PERARD
- use MATCH_OPTION()
- move arm_sci struct and cfg params in "arch_arm"
- add SCMI passthrough for dom0less case
- toolstack comments from Anthony PERARD
- added dom0less support
- added doc for "xen,scmi-secondary-agents"
Grygorii Strashko (6):
xen/arm: scmi-smc: update to be used under sci subsystem
xen/arm: scmi-smc: passthrough SCMI SMC to domain, single agent
docs: arm: add docs for SCMI over SMC calls forwarding driver
xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
docs: arm: add SCI SCMI SMC multi-agent driver docs
docs: arm: proposal to add separate SCMI node for Xen agent
Oleksii Moisieiev (2):
xen/arm: add generic SCI subsystem
xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
MAINTAINERS | 6 +
.../arm/firmware/arm-scmi-proposal.rst | 224 +++++
.../arm/firmware/arm-scmi.rst | 442 +++++++++
docs/hypervisor-guide/arm/index.rst | 9 +
docs/hypervisor-guide/index.rst | 1 +
docs/man/xl.cfg.5.pod.in | 47 +
docs/misc/arm/device-tree/booting.txt | 75 ++
docs/misc/xen-command-line.pandoc | 18 +
tools/include/libxl.h | 5 +
tools/libs/light/libxl_arm.c | 18 +
tools/libs/light/libxl_types.idl | 12 +
tools/xl/xl_parse.c | 48 +
xen/arch/arm/device.c | 5 +
xen/arch/arm/dom0less-build.c | 49 +
xen/arch/arm/domain.c | 12 +-
xen/arch/arm/domain_build.c | 11 +-
xen/arch/arm/firmware/Kconfig | 36 +-
xen/arch/arm/firmware/Makefile | 2 +
xen/arch/arm/firmware/sci.c | 191 ++++
xen/arch/arm/firmware/scmi-proto.h | 164 ++++
xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
xen/arch/arm/firmware/scmi-shmem.h | 45 +
xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++
xen/arch/arm/firmware/scmi-smc.c | 191 +++-
xen/arch/arm/include/asm/domain.h | 5 +
xen/arch/arm/include/asm/firmware/sci.h | 214 +++++
xen/arch/arm/include/asm/firmware/scmi-smc.h | 41 -
xen/arch/arm/vsmc.c | 4 +-
xen/common/domctl.c | 19 +
xen/drivers/passthrough/device_tree.c | 6 +
xen/include/asm-generic/device.h | 1 +
xen/include/public/arch-arm.h | 8 +
32 files changed, 2856 insertions(+), 86 deletions(-)
create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi.rst
create mode 100644 docs/hypervisor-guide/arm/index.rst
create mode 100644 xen/arch/arm/firmware/sci.c
create mode 100644 xen/arch/arm/firmware/scmi-proto.h
create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
create mode 100644 xen/arch/arm/include/asm/firmware/sci.h
delete mode 100644 xen/arch/arm/include/asm/firmware/scmi-smc.h
--
2.34.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* [RFC PATCH v4 2/8] xen/arm: scmi-smc: update to be used under sci subsystem
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-19 23:29 ` Stefano Stabellini
2025-05-19 15:50 ` [RFC PATCH v4 1/8] xen/arm: add generic SCI subsystem Oleksii Moisieiev
` (7 subsequent siblings)
8 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
From: Grygorii Strashko <grygorii_strashko@epam.com>
The introduced SCI (System Control Interface) subsystem provides unified
interface to integrate in Xen SCI drivers which adds support for ARM
firmware (EL3, SCP) based software interfaces (like SCMI) that are used in
system management. The SCI subsystem allows to add drivers for different FW
interfaces or have different drivers for the same FW interface (for example,
SCMI with different transports).
This patch updates SCMI over SMC calls handling layer, introduced by
commit 3e322bef8bc0 ("xen/arm: firmware: Add SCMI over SMC calls handling
layer"), to be SCI driver:
- convert to DT device;
- convert to SCI Xen interface.
There are no functional changes in general, the driver is just adopted
to the SCI interface.
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
---
xen/arch/arm/firmware/Kconfig | 13 ++-
xen/arch/arm/firmware/scmi-smc.c | 93 +++++++++++---------
xen/arch/arm/include/asm/firmware/scmi-smc.h | 41 ---------
xen/arch/arm/vsmc.c | 5 +-
xen/include/public/arch-arm.h | 1 +
5 files changed, 64 insertions(+), 89 deletions(-)
delete mode 100644 xen/arch/arm/include/asm/firmware/scmi-smc.h
diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
index fc7918c7fc..bbf88fbb9a 100644
--- a/xen/arch/arm/firmware/Kconfig
+++ b/xen/arch/arm/firmware/Kconfig
@@ -8,9 +8,18 @@ config ARM_SCI
menu "Firmware Drivers"
+choice
+ prompt "ARM SCI driver type"
+ default SCMI_SMC
+ help
+ Choose which ARM SCI driver to enable.
+
+config ARM_SCI_NONE
+ bool "none"
+
config SCMI_SMC
bool "Forward SCMI over SMC calls from hwdom to EL3 firmware"
- default y
+ select ARM_SCI
help
This option enables basic awareness for SCMI calls using SMC as
doorbell mechanism and Shared Memory for transport ("arm,scmi-smc"
@@ -18,4 +27,6 @@ config SCMI_SMC
firmware node is used to trap and forward corresponding SCMI SMCs
to firmware running at EL3, for calls coming from the hardware domain.
+endchoice
+
endmenu
diff --git a/xen/arch/arm/firmware/scmi-smc.c b/xen/arch/arm/firmware/scmi-smc.c
index 33473c04b1..13d1137592 100644
--- a/xen/arch/arm/firmware/scmi-smc.c
+++ b/xen/arch/arm/firmware/scmi-smc.c
@@ -9,6 +9,7 @@
* Copyright 2024 NXP
*/
+#include <asm/device.h>
#include <xen/acpi.h>
#include <xen/device_tree.h>
#include <xen/errno.h>
@@ -16,12 +17,11 @@
#include <xen/sched.h>
#include <xen/types.h>
+#include <asm/firmware/sci.h>
#include <asm/smccc.h>
-#include <asm/firmware/scmi-smc.h>
#define SCMI_SMC_ID_PROP "arm,smc-id"
-static bool __ro_after_init scmi_enabled;
static uint32_t __ro_after_init scmi_smc_id;
/*
@@ -41,14 +41,11 @@ static bool scmi_is_valid_smc_id(uint32_t fid)
*
* Returns true if SMC was handled (regardless of response), false otherwise.
*/
-bool scmi_handle_smc(struct cpu_user_regs *regs)
+static bool scmi_handle_smc(struct cpu_user_regs *regs)
{
uint32_t fid = (uint32_t)get_user_reg(regs, 0);
struct arm_smccc_res res;
- if ( !scmi_enabled )
- return false;
-
if ( !scmi_is_valid_smc_id(fid) )
return false;
@@ -78,49 +75,45 @@ bool scmi_handle_smc(struct cpu_user_regs *regs)
return true;
}
-static int __init scmi_check_smccc_ver(void)
+static int scmi_smc_domain_init(struct domain *d,
+ struct xen_domctl_createdomain *config)
{
- if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
- {
- printk(XENLOG_WARNING
- "SCMI: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
- return -ENOSYS;
- }
+ if ( !is_hardware_domain(d) )
+ return 0;
+ d->arch.sci_enabled = true;
+ printk(XENLOG_DEBUG "SCMI: %pd init\n", d);
return 0;
}
-static int __init scmi_dt_init_smccc(void)
+static void scmi_smc_domain_destroy(struct domain *d)
{
- static const struct dt_device_match scmi_ids[] __initconst =
- {
- /* We only support "arm,scmi-smc" binding for now */
- DT_MATCH_COMPATIBLE("arm,scmi-smc"),
- { /* sentinel */ },
- };
- const struct dt_device_node *scmi_node;
- int ret;
+ if ( !is_hardware_domain(d) )
+ return;
- /* If no SCMI firmware node found, fail silently as it's not mandatory */
- scmi_node = dt_find_matching_node(NULL, scmi_ids);
- if ( !scmi_node )
- return -EOPNOTSUPP;
+ printk(XENLOG_DEBUG "SCMI: %pd destroy\n", d);
+}
- ret = dt_property_read_u32(scmi_node, SCMI_SMC_ID_PROP, &scmi_smc_id);
- if ( !ret )
+static int __init scmi_check_smccc_ver(void)
+{
+ if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
{
- printk(XENLOG_ERR "SCMI: No valid \"%s\" property in \"%s\" DT node\n",
- SCMI_SMC_ID_PROP, scmi_node->full_name);
- return -ENOENT;
+ printk(XENLOG_WARNING
+ "SCMI: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
+ return -ENOSYS;
}
- scmi_enabled = true;
-
return 0;
}
+static const struct sci_mediator_ops scmi_smc_ops = {
+ .handle_call = scmi_handle_smc,
+ .domain_init = scmi_smc_domain_init,
+ .domain_destroy = scmi_smc_domain_destroy,
+};
+
/* Initialize the SCMI layer based on SMCs and Device-tree */
-static int __init scmi_init(void)
+static int __init scmi_dom0_init(struct dt_device_node *dev, const void *data)
{
int ret;
@@ -134,22 +127,36 @@ static int __init scmi_init(void)
if ( ret )
return ret;
- ret = scmi_dt_init_smccc();
- if ( ret == -EOPNOTSUPP )
- return ret;
+ ret = dt_property_read_u32(dev, SCMI_SMC_ID_PROP, &scmi_smc_id);
+ if ( !ret )
+ {
+ printk(XENLOG_ERR "SCMI: No valid \"%s\" property in \"%s\" DT node\n",
+ SCMI_SMC_ID_PROP, dt_node_full_name(dev));
+ return -ENOENT;
+ }
+
+ ret = sci_register(&scmi_smc_ops);
if ( ret )
- goto err;
+ {
+ printk(XENLOG_ERR "SCMI: mediator already registered (ret = %d)\n",
+ ret);
+ return ret;
+ }
printk(XENLOG_INFO "Using SCMI with SMC ID: 0x%x\n", scmi_smc_id);
return 0;
-
- err:
- printk(XENLOG_ERR "SCMI: Initialization failed (ret = %d)\n", ret);
- return ret;
}
-__initcall(scmi_init);
+static const struct dt_device_match scmi_smc_match[] __initconst = {
+ DT_MATCH_COMPATIBLE("arm,scmi-smc"),
+ { /* sentinel */ },
+};
+
+DT_DEVICE_START(scmi_smc, "SCMI SMC DOM0", DEVICE_FIRMWARE)
+ .dt_match = scmi_smc_match,
+ .init = scmi_dom0_init,
+DT_DEVICE_END
/*
* Local variables:
diff --git a/xen/arch/arm/include/asm/firmware/scmi-smc.h b/xen/arch/arm/include/asm/firmware/scmi-smc.h
deleted file mode 100644
index 6b1a164a40..0000000000
--- a/xen/arch/arm/include/asm/firmware/scmi-smc.h
+++ /dev/null
@@ -1,41 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * xen/arch/arm/include/asm/firmware/scmi-smc.h
- *
- * ARM System Control and Management Interface (SCMI) over SMC
- * Generic handling layer
- *
- * Andrei Cherechesu <andrei.cherechesu@nxp.com>
- * Copyright 2024 NXP
- */
-
-#ifndef __ASM_SCMI_SMC_H__
-#define __ASM_SCMI_SMC_H__
-
-#include <xen/types.h>
-
-struct cpu_user_regs;
-
-#ifdef CONFIG_SCMI_SMC
-
-bool scmi_handle_smc(struct cpu_user_regs *regs);
-
-#else
-
-static inline bool scmi_handle_smc(struct cpu_user_regs *regs)
-{
- return false;
-}
-
-#endif /* CONFIG_SCMI_SMC */
-
-#endif /* __ASM_SCMI_H__ */
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index 51b3c02973..b33c69a1c2 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -21,7 +21,6 @@
#include <asm/traps.h>
#include <asm/vpsci.h>
#include <asm/platform.h>
-#include <asm/firmware/scmi-smc.h>
/* Number of functions currently supported by Hypervisor Service. */
#define XEN_SMCCC_FUNCTION_COUNT 3
@@ -233,7 +232,7 @@ static bool handle_sip(struct cpu_user_regs *regs)
if ( platform_smc(regs) )
return true;
- return scmi_handle_smc(regs);
+ return sci_handle_call(regs);
}
/*
@@ -301,8 +300,6 @@ static bool vsmccc_handle_call(struct cpu_user_regs *regs)
break;
case ARM_SMCCC_OWNER_SIP:
handled = handle_sip(regs);
- if ( !handled )
- handled = sci_handle_call(regs);
break;
case ARM_SMCCC_OWNER_TRUSTED_APP ... ARM_SMCCC_OWNER_TRUSTED_APP_END:
case ARM_SMCCC_OWNER_TRUSTED_OS ... ARM_SMCCC_OWNER_TRUSTED_OS_END:
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 55eed9992c..095b1a23e3 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -328,6 +328,7 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
#define XEN_DOMCTL_CONFIG_TEE_FFA 2
#define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
+#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC 1
struct xen_arch_domainconfig {
/* IN/OUT */
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH v4 1/8] xen/arm: add generic SCI subsystem
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 2/8] xen/arm: scmi-smc: update to be used under sci subsystem Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-19 23:45 ` Stefano Stabellini
2025-05-19 15:50 ` [RFC PATCH v4 3/8] xen/arm: scmi-smc: passthrough SCMI SMC to domain, single agent Oleksii Moisieiev
` (6 subsequent siblings)
8 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
This patch adds the basic framework for ARM SCI mediator. SCI is System
Control Interface, which is designed to redirect requests from the Domains
to ARM specific Firmware (for example SCMI). This will allow the devices,
passed-through to the different Domains, to access to the System resources
(such as clocks/resets etc) by sending requests to the firmware.
ARM SCI subsystem allows to implement different SCI drivers to handle
specific ARM firmware interfaces (like ARM SCMI) and mediate requests
between the Domains and the Firmware. Also it allows SCI drivers to perform
proper action during Domain creation/destruction which is vital for
handling use cases like Domain reboot.
This patch introduces new DEVICE_FIRMWARE device subclass for probing SCI
drivers basing on device tree, SCI drivers register itself with
DT_DEVICE_START/END macro. On init - the SCI drivers should register its
SCI ops with sci_register(). Only one SCI driver can be supported.
At run-time, the following SCI API calls are introduced:
- sci_domain_sanitise_config() called from arch_sanitise_domain_config()
- sci_domain_init() called from arch_domain_create()
- sci_relinquish_resources() called from domain_relinquish_resources()
- sci_domain_destroy() called from arch_domain_destroy()
- sci_handle_call() called from vsmccc_handle_call()
- sci_dt_handle_node()
sci_dt_finalize() called from handle_node() (Dom0 DT)
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
Changes in v4:
- fix SPDX-License
- rename DEVICE_ARM_SCI DT device class to FIRMWARE_DEVICE
- move XEN_DOMCTL_assign_device code in separate patch
- Add documentation for SCI SCMI drivers
MAINTAINERS | 6 +
xen/arch/arm/device.c | 5 +
xen/arch/arm/dom0less-build.c | 7 +
xen/arch/arm/domain.c | 12 +-
xen/arch/arm/domain_build.c | 8 +
xen/arch/arm/firmware/Kconfig | 8 +
xen/arch/arm/firmware/Makefile | 1 +
xen/arch/arm/firmware/sci.c | 154 ++++++++++++++++++
xen/arch/arm/include/asm/domain.h | 5 +
xen/arch/arm/include/asm/firmware/sci.h | 200 ++++++++++++++++++++++++
xen/arch/arm/vsmc.c | 3 +
xen/include/asm-generic/device.h | 1 +
xen/include/public/arch-arm.h | 4 +
13 files changed, 413 insertions(+), 1 deletion(-)
create mode 100644 xen/arch/arm/firmware/sci.c
create mode 100644 xen/arch/arm/include/asm/firmware/sci.h
diff --git a/MAINTAINERS b/MAINTAINERS
index c11b82eca9..f5e3c48b96 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -507,6 +507,12 @@ R: George Dunlap <gwd@xenproject.org>
S: Supported
F: xen/common/sched/
+SCI MEDIATORS
+M: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
+S: Supported
+F: xen/arch/arm/firmware/sci.c
+F: xen/arch/arm/include/asm/firmware/sci.h
+
SEABIOS UPSTREAM
M: Wei Liu <wl@xen.org>
S: Supported
diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index 5610cddcba..bdab96a408 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -13,6 +13,7 @@
#include <xen/iocap.h>
#include <xen/lib.h>
+#include <asm/firmware/sci.h>
#include <asm/setup.h>
int map_irq_to_domain(struct domain *d, unsigned int irq,
@@ -303,6 +304,10 @@ int handle_device(struct domain *d, struct dt_device_node *dev, p2m_type_t p2mt,
return res;
}
}
+
+ res = sci_assign_dt_device(d, dev);
+ if ( res )
+ return res;
}
res = map_device_irqs_to_domain(d, dev, own_device, irq_ranges);
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 49d1f14d65..a09c4c4bd7 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -14,6 +14,7 @@
#include <asm/arm64/sve.h>
#include <asm/dom0less-build.h>
#include <asm/domain_build.h>
+#include <asm/firmware/sci.h>
#include <asm/static-memory.h>
#include <asm/static-shmem.h>
@@ -321,6 +322,10 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo,
return -EINVAL;
}
+ res = sci_assign_dt_device(kinfo->d, node);
+ if ( res )
+ return res;
+
res = map_device_irqs_to_domain(kinfo->d, node, true, NULL);
if ( res < 0 )
return res;
@@ -970,6 +975,8 @@ void __init create_domUs(void)
if ( !llc_coloring_enabled && llc_colors_str )
panic("'llc-colors' found, but LLC coloring is disabled\n");
+ d_cfg.arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
+
/*
* The variable max_init_domid is initialized with zero, so here it's
* very important to use the pre-increment operator to call
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 3ba959f866..652aeb7a55 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -25,6 +25,7 @@
#include <asm/platform.h>
#include <asm/procinfo.h>
#include <asm/regs.h>
+#include <asm/firmware/sci.h>
#include <asm/tee/tee.h>
#include <asm/vfp.h>
#include <asm/vgic.h>
@@ -694,7 +695,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
return -EINVAL;
}
- return 0;
+ return sci_domain_sanitise_config(config);
}
int arch_domain_create(struct domain *d,
@@ -786,6 +787,9 @@ int arch_domain_create(struct domain *d,
d->arch.sve_vl = config->arch.sve_vl;
#endif
+ if ( (rc = sci_domain_init(d, config)) != 0 )
+ goto fail;
+
return 0;
fail:
@@ -846,6 +850,7 @@ void arch_domain_destroy(struct domain *d)
domain_vgic_free(d);
domain_vuart_free(d);
free_xenheap_page(d->shared_info);
+ sci_domain_destroy(d);
#ifdef CONFIG_ACPI
free_xenheap_pages(d->arch.efi_acpi_table,
get_order_from_bytes(d->arch.efi_acpi_len));
@@ -1039,6 +1044,7 @@ enum {
PROG_p2m_root,
PROG_p2m,
PROG_p2m_pool,
+ PROG_sci,
PROG_done,
};
@@ -1098,6 +1104,10 @@ int domain_relinquish_resources(struct domain *d)
ret = relinquish_p2m_mapping(d);
if ( ret )
return ret;
+ PROGRESS(sci):
+ ret = sci_relinquish_resources(d);
+ if ( ret )
+ return ret;
PROGRESS(p2m_root):
/*
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 7b47abade1..36d28b52a4 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -24,6 +24,7 @@
#include <asm/setup.h>
#include <asm/tee/tee.h>
#include <asm/pci.h>
+#include <asm/firmware/sci.h>
#include <asm/platform.h>
#include <asm/psci.h>
#include <asm/setup.h>
@@ -1888,6 +1889,9 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
return 0;
}
+ if ( sci_dt_handle_node(d, node) )
+ return 0;
+
/*
* The vGIC does not support routing hardware PPIs to guest. So
* we need to skip any node using PPIs.
@@ -1988,6 +1992,10 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
if ( res )
return res;
+ res = sci_dt_finalize(d, kinfo->fdt);
+ if ( res )
+ return res;
+
/*
* Create a second memory node to store the ranges covering
* reserved-memory regions.
diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
index 817da745fd..fc7918c7fc 100644
--- a/xen/arch/arm/firmware/Kconfig
+++ b/xen/arch/arm/firmware/Kconfig
@@ -1,3 +1,11 @@
+config ARM_SCI
+ bool
+ depends on ARM
+ help
+ This option enables generic Arm SCI (System Control Interface) mediators
+ support. It allows domains to control system resources via one of
+ Arm SCI mediators drivers implemented in XEN, like SCMI.
+
menu "Firmware Drivers"
config SCMI_SMC
diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
index a5e4542666..71bdefc24a 100644
--- a/xen/arch/arm/firmware/Makefile
+++ b/xen/arch/arm/firmware/Makefile
@@ -1 +1,2 @@
+obj-$(CONFIG_ARM_SCI) += sci.o
obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
new file mode 100644
index 0000000000..e1522e10e2
--- /dev/null
+++ b/xen/arch/arm/firmware/sci.c
@@ -0,0 +1,154 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Generic part of the SCI (System Control Interface) subsystem.
+ *
+ * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
+ * Copyright (c) 2025 EPAM Systems
+ */
+
+#include <xen/acpi.h>
+#include <xen/errno.h>
+#include <xen/init.h>
+#include <xen/sched.h>
+#include <xen/types.h>
+
+#include <asm/firmware/sci.h>
+
+static const struct sci_mediator_ops __read_mostly *cur_mediator;
+
+int sci_register(const struct sci_mediator_ops *ops)
+{
+ if ( cur_mediator )
+ return -EEXIST;
+
+ if ( !ops->domain_init || !ops->domain_destroy || !ops->handle_call )
+ return -EINVAL;
+
+ cur_mediator = ops;
+
+ return 0;
+};
+
+bool sci_handle_call(struct cpu_user_regs *args)
+{
+ if ( unlikely(!cur_mediator) )
+ return false;
+
+ return cur_mediator->handle_call(args);
+}
+
+int sci_domain_init(struct domain *d, struct xen_domctl_createdomain *config)
+{
+ if ( !cur_mediator )
+ return 0;
+
+ return cur_mediator->domain_init(d, config);
+}
+
+int sci_domain_sanitise_config(struct xen_domctl_createdomain *config)
+{
+ if ( !cur_mediator )
+ return 0;
+
+ if ( !cur_mediator->domain_sanitise_config )
+ return 0;
+
+ return cur_mediator->domain_sanitise_config(config);
+}
+
+void sci_domain_destroy(struct domain *d)
+{
+ if ( !cur_mediator )
+ return;
+
+ cur_mediator->domain_destroy(d);
+}
+
+int sci_relinquish_resources(struct domain *d)
+{
+ if ( !cur_mediator )
+ return 0;
+
+ if ( !cur_mediator->relinquish_resources )
+ return 0;
+
+ return cur_mediator->relinquish_resources(d);
+}
+
+bool sci_dt_handle_node(struct domain *d, struct dt_device_node *node)
+{
+ if ( !cur_mediator )
+ return 0;
+
+ if ( !cur_mediator->dom0_dt_handle_node )
+ return 0;
+
+ return cur_mediator->dom0_dt_handle_node(d, node);
+}
+
+int sci_dt_finalize(struct domain *d, void *fdt)
+{
+ if ( !cur_mediator )
+ return 0;
+
+ if ( !cur_mediator->dom0_dt_finalize )
+ return 0;
+
+ return cur_mediator->dom0_dt_finalize(d, fdt);
+}
+
+int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
+{
+ struct dt_phandle_args ac_spec;
+ int index = 0;
+ int ret;
+
+ if ( !cur_mediator )
+ return 0;
+
+ if ( !cur_mediator->assign_dt_device )
+ return 0;
+
+ while ( !dt_parse_phandle_with_args(dev, "access-controllers",
+ "#access-controller-cells", index,
+ &ac_spec) )
+ {
+ printk(XENLOG_DEBUG "sci: assign device %s to %pd\n",
+ dt_node_full_name(dev), d);
+
+ ret = cur_mediator->assign_dt_device(d, &ac_spec);
+ if ( ret )
+ return ret;
+
+ index++;
+ }
+
+ return 0;
+}
+
+static int __init sci_init(void)
+{
+ struct dt_device_node *np;
+ unsigned int num_sci = 0;
+ int rc;
+
+ dt_for_each_device_node(dt_host, np)
+ {
+ rc = device_init(np, DEVICE_FIRMWARE, NULL);
+ if ( !rc && num_sci )
+ {
+ printk(XENLOG_ERR
+ "SCMI: Only one SCI controller is supported. found second %s\n",
+ np->name);
+ return -EOPNOTSUPP;
+ }
+ else if ( !rc )
+ num_sci++;
+ else if ( rc != -EBADF && rc != -ENODEV )
+ return rc;
+ }
+
+ return 0;
+}
+
+__initcall(sci_init);
diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
index f1d72c6e48..fa0898b7cf 100644
--- a/xen/arch/arm/include/asm/domain.h
+++ b/xen/arch/arm/include/asm/domain.h
@@ -118,6 +118,11 @@ struct arch_domain
#ifdef CONFIG_TEE
void *tee;
#endif
+#ifdef CONFIG_ARM_SCI
+ bool sci_enabled;
+ /* ARM SCI driver's specific data */
+ void *sci_data;
+#endif
} __cacheline_aligned;
diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
new file mode 100644
index 0000000000..71fb54852e
--- /dev/null
+++ b/xen/arch/arm/include/asm/firmware/sci.h
@@ -0,0 +1,200 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Generic ARM SCI (System Control Interface) subsystem.
+ *
+ * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
+ * Copyright (c) 2025 EPAM Systems
+ */
+
+#ifndef __ASM_ARM_SCI_H
+#define __ASM_ARM_SCI_H
+
+#include <xen/lib.h>
+#include <xen/types.h>
+#include <xen/device_tree.h>
+#include <xen/errno.h>
+#include <xen/sched.h>
+
+#ifdef CONFIG_ARM_SCI
+
+struct sci_mediator_ops {
+ /*
+ * Called during domain construction. If it is requested to enable
+ * SCI support, so SCI driver can create own structures for the new domain
+ * and inform firmware about new domain (if required).
+ * Mandatory.
+ */
+ int (*domain_init)(struct domain *d,
+ struct xen_domctl_createdomain *config);
+
+ /*
+ * Called during domain construction. The SCI driver uses
+ * it to sanitize domain SCI configuration parameters.
+ * Optional.
+ */
+ int (*domain_sanitise_config)(struct xen_domctl_createdomain *config);
+
+ /*
+ * Called during domain destruction, releases all resources, that
+ * were allocated for domain.
+ * Mandatory.
+ */
+ void (*domain_destroy)(struct domain *d);
+
+ /*
+ * Called during domain destruction to relinquish resources used
+ * by SCI driver itself and request resources releasing from firmware.
+ * Optional.
+ */
+ int (*relinquish_resources)(struct domain *d);
+
+ /* SMC/HVC Handle callback */
+ bool (*handle_call)(struct cpu_user_regs *regs);
+
+ /*
+ * Dom0 DT nodes handling callback so SCI driver can detect DT nodes it
+ * need to handle and decide if those nodes need to be provided to Dom0.
+ * Optional.
+ */
+ bool (*dom0_dt_handle_node)(struct domain *d, struct dt_device_node *node);
+
+ /*
+ * SCI driver callback called at the end of Dom0 DT generation, so
+ * it can perform steps to modify DT to enable/disable SCI
+ * functionality for Dom0.
+ */
+ int (*dom0_dt_finalize)(struct domain *d, void *fdt);
+
+ /*
+ * SCI driver callback called when DT device is passed through to guest,
+ * so SCI driver can enable device access to the domain if SCI FW provides
+ * Device specific access control functionality.
+ * Optional.
+ */
+ int (*assign_dt_device)(struct domain *d, struct dt_phandle_args *ac_spec);
+};
+
+
+static inline bool sci_domain_is_enabled(struct domain *d)
+{
+ return d->arch.sci_enabled;
+}
+
+/*
+ * Register SCI subsystem ops.
+ *
+ * Register SCI drivers operation and so enable SCI functionality.
+ * Only one SCI driver is supported.
+ */
+int sci_register(const struct sci_mediator_ops *ops);
+
+/*
+ * Initialize SCI functionality for domain if configured.
+ *
+ * Initialization routine to enable SCI functionality for the domain.
+ * The SCI configuration data and decision about enabling SCI functionality
+ * for the domain is SCI driver specific.
+ */
+int sci_domain_init(struct domain *d, struct xen_domctl_createdomain *config);
+
+/*
+ * Sanitise domain configuration parameters.
+ *
+ */
+int sci_domain_sanitise_config(struct xen_domctl_createdomain *config);
+
+/*
+ * Destroy SCI domain instance.
+ */
+void sci_domain_destroy(struct domain *d);
+
+/*
+ * Free resources assigned to the certain domain.
+ */
+int sci_relinquish_resources(struct domain *d);
+
+/*
+ * SMC/HVC Handle callback.
+ *
+ * SCI driver acts as SMC/HVC server for the registered domains and
+ * does redirection of the domain calls to the SCI firmware,
+ * such as ARM TF-A or similar.
+ */
+bool sci_handle_call(struct cpu_user_regs *regs);
+
+/*
+ * Dom0 DT nodes handling function.
+ *
+ * Allows SCI driver to detect DT nodes it need to handle and decide if
+ * those nodes need to be provided to Dom0.
+ */
+bool sci_dt_handle_node(struct domain *d, struct dt_device_node *node);
+
+/*
+ * Dom0 DT generation finalize.
+ *
+ * Called at the end of Dom0 DT generation, so SCI driver can perform steps
+ * to modify DT to enable/disable SCI functionality for Dom0.
+ */
+int sci_dt_finalize(struct domain *d, void *fdt);
+
+/*
+ * Assign DT device to domain.
+ *
+ * Called when DT device is passed through to guest, so SCI driver can enable
+ * device access to the domain if SCI FW provides "Device specific access
+ * control" functionality.
+ */
+int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
+#else
+
+static inline bool sci_domain_is_enabled(struct domain *d)
+{
+ return false;
+}
+
+static inline int sci_domain_init(struct domain *d,
+ struct xen_domctl_createdomain *config)
+{
+ return 0;
+}
+
+static inline int
+sci_domain_sanitise_config(struct xen_domctl_createdomain *config)
+{
+ return 0;
+}
+
+static inline void sci_domain_destroy(struct domain *d)
+{}
+
+static inline int sci_relinquish_resources(struct domain *d)
+{
+ return 0;
+}
+
+static inline bool sci_handle_call(struct cpu_user_regs *args)
+{
+ return false;
+}
+
+static inline bool sci_dt_handle_node(struct domain *d,
+ struct dt_device_node *node)
+{
+ return false;
+}
+
+static inline int sci_dt_finalize(struct domain *d, void *fdt)
+{
+ return false;
+}
+
+static inline int sci_assign_dt_device(struct domain *d,
+ struct dt_device_node *dev)
+{
+ return 0;
+}
+
+#endif /* CONFIG_ARM_SCI */
+
+#endif /* __ASM_ARM_SCI_H */
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index 62d8117a12..51b3c02973 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -12,6 +12,7 @@
#include <public/arch-arm/smccc.h>
#include <asm/cpuerrata.h>
#include <asm/cpufeature.h>
+#include <asm/firmware/sci.h>
#include <asm/monitor.h>
#include <asm/regs.h>
#include <asm/smccc.h>
@@ -300,6 +301,8 @@ static bool vsmccc_handle_call(struct cpu_user_regs *regs)
break;
case ARM_SMCCC_OWNER_SIP:
handled = handle_sip(regs);
+ if ( !handled )
+ handled = sci_handle_call(regs);
break;
case ARM_SMCCC_OWNER_TRUSTED_APP ... ARM_SMCCC_OWNER_TRUSTED_APP_END:
case ARM_SMCCC_OWNER_TRUSTED_OS ... ARM_SMCCC_OWNER_TRUSTED_OS_END:
diff --git a/xen/include/asm-generic/device.h b/xen/include/asm-generic/device.h
index 1acd1ba1d8..e96c5558c2 100644
--- a/xen/include/asm-generic/device.h
+++ b/xen/include/asm-generic/device.h
@@ -18,6 +18,7 @@ enum device_class
DEVICE_IOMMU,
DEVICE_INTERRUPT_CONTROLLER,
DEVICE_PCI_HOSTBRIDGE,
+ DEVICE_FIRMWARE,
/* Use for error */
DEVICE_UNKNOWN,
};
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 24840eeaa6..55eed9992c 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -327,6 +327,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
#define XEN_DOMCTL_CONFIG_TEE_OPTEE 1
#define XEN_DOMCTL_CONFIG_TEE_FFA 2
+#define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
+
struct xen_arch_domainconfig {
/* IN/OUT */
uint8_t gic_version;
@@ -350,6 +352,8 @@ struct xen_arch_domainconfig {
*
*/
uint32_t clock_frequency;
+ /* IN */
+ uint8_t arm_sci_type;
};
#endif /* __XEN__ || __XEN_TOOLS__ */
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH v4 3/8] xen/arm: scmi-smc: passthrough SCMI SMC to domain, single agent
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 2/8] xen/arm: scmi-smc: update to be used under sci subsystem Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 1/8] xen/arm: add generic SCI subsystem Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-20 0:18 ` Stefano Stabellini
2025-05-19 15:50 ` [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu Oleksii Moisieiev
` (5 subsequent siblings)
8 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
From: Grygorii Strashko <grygorii_strashko@epam.com>
The commit 3e322bef8bc0 ("xen/arm: firmware: Add SCMI over SMC calls
handling layer") introduces simple driver which forwards SCMI over SMC
calls from hwdom/dom0 to EL3 firmware (TF-A) with a single SCMI OSPM agent
support. While it is working gracefully for hwdom/dom0 use case it doesn't
cover "thin Dom0 with guest domain, which serves as Driver domain"
use-case. In this case HW need to be enable in Driver domain and dom0 is
performing only control functions.
The EL3 SCMI firmware (TF-A) with a single SCMI OSPM agent support is
pretty generic case for the default vendors SDK and new platforms.
This patch enables passthrough of SCMI SMC single agent interface to the
one guest domain serving as Driver domain.
Configure Dom0 to enable SCMI passthrough:
- dom0: add dom0_scmi_smc_passthrough to the Xen Command Line
Enabled SCMI passthrough for guest using toolstack in the following way:
- domD: xl.cfg add "arm_sci" option as below
arm_sci = "type=scmi_smc"
- domD: xl.cfg enable access to the "arm,scmi-shmem"
iomem = [
"47ff0,1@22001",
]
- domD: add SCMI nodes to the Driver domain partial device tree as in the
below example:
passthrough {
scmi_shm_0: sram@22001000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x22001000 0x0 0x1000>;
};
firmware {
compatible = "simple-bus";
scmi: scmi {
compatible = "arm,scmi-smc";
shmem = <&scmi_shm_0>;
...
}
}
}
dom0less case configuration:
- add "xen,sci_type" property for required DomU ("xen,domain") node
xen,sci_type="scmi_smc"
- add scmi nodes to the Driver domain partial device tree the same way
as above and enable access to the "arm,scmi-shmem" according to
dom0less documentation. For example:
scmi_shm_0: sram@22001000 {
compatible = "arm,scmi-shmem";
reg = <0x00 0x22001000 0x00 0x1000>;
-> xen,reg = <0x0 0x47ff0000 0x0 0x1000 0x0 0x22001000>;
-> xen,force-assign-without-iommu;
};
The SCMI SMC single agent interface can be enabled for one and only one
domain. In general, the configuration is similar to any other HW
passthrough, except explicitly enabling SCMI with "arm_sci" xl.cfg option.
Note that "arm,scmi-smc" and "arm,scmi-shmem" nodes will be removed from
dom0/hwdom DT in case of
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
---
Changes in v4:
- xl.cfg doc
- fix comments from Stefano Stabellini
- fix toolstack code as sugested by Anthony PERARD
- use MATCH_OPTION()
- move arm_sci struct and cfg params in "arch_arm"
- add SCMI passthrough for dom0less case
docs/man/xl.cfg.5.pod.in | 34 ++++++++
docs/misc/arm/device-tree/booting.txt | 15 ++++
docs/misc/xen-command-line.pandoc | 9 +++
tools/include/libxl.h | 5 ++
tools/libs/light/libxl_arm.c | 14 ++++
tools/libs/light/libxl_types.idl | 10 +++
tools/xl/xl_parse.c | 36 +++++++++
xen/arch/arm/dom0less-build.c | 33 +++++++-
xen/arch/arm/firmware/Kconfig | 4 +-
xen/arch/arm/firmware/scmi-smc.c | 112 +++++++++++++++++++++++++-
10 files changed, 267 insertions(+), 5 deletions(-)
diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8e1422104e..1ccf50b8ea 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3092,6 +3092,40 @@ Otherwise, the value specified by the `nr_spis` parameter will be used.
The number of SPIs should match the highest interrupt ID that will be
assigned to the domain.
+=item B<arm_sci="ARM_SCI_STRING">
+
+Set ARM_SCI specific options for the guest. ARM SCI is System
+Control Protocol allows domain to manage various functions that are provided
+by HW platform firmware.
+
+B<ARM_SCI_STRING> is a comma separated list of C<KEY=VALUE> settings,
+from the following list:
+
+=over 4
+
+=item B<type=STRING>
+
+Specifies an ARM SCI type for the guest.
+
+=over 4
+
+=item B<none>
+
+Don't allow guest to use ARM SCI if present on the platform. This is the
+default value.
+
+=item B<scmi_smc>
+
+Enables ARM SCMI SMC support for the guest by enabling SCMI over SMC calls
+forwarding from domain to the EL3 firmware (like Trusted Firmware-A) with a
+single SCMI OSPM agent support.
+Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
+option.
+
+=back
+
+=back
+
=back
=head3 x86
diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 9c881baccc..8943c04173 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -281,6 +281,21 @@ with the following properties:
passed through. This option is the default if this property is missing
and the user does not provide the device partial device tree for the domain.
+- xen,sci_type
+
+ A string property specifying an ARM SCI type for the guest.
+
+ - "none"
+ Don't allow guest to use ARM SCI if present on the platform. This is the
+ default value.
+
+ - "scmi_smc"
+ Enables ARM SCMI SMC support for the guest by enabling SCMI over SMC calls
+ forwarding from domain to the EL3 firmware (like Trusted Firmware-A) with a
+ single SCMI OSPM agent support.
+ Should be used together with dom0_scmi_smc_passthrough Xen command line
+ option.
+
Under the "xen,domain" compatible node, one or more sub-nodes are present
for the DomU kernel and ramdisk.
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 9bbd00baef..8e50f6b7c7 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1082,6 +1082,15 @@ affinities to prefer but be not limited to the specified node(s).
Pin dom0 vcpus to their respective pcpus
+### dom0_scmi_smc_passthrough (ARM)
+> `= <boolean>`
+
+The option is available when `CONFIG_SCMI_SMC` is compiled in, and allows to
+enable SCMI SMC single agent interface for any, but only one guest domain,
+which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
+SCMI nodes removed from Dom0/hwdom device tree.
+(for example, thin Dom0 with Driver domain use-case).
+
### dtuart (ARM)
> `= path [:options]`
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index f8fe4afd7d..5fa43637ab 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -313,6 +313,11 @@
*/
#define LIBXL_HAVE_BUILDINFO_ARCH_NR_SPIS 1
+/*
+ * libxl_domain_build_info has the arch_arm.sci* fields.
+ */
+#define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SCI 1
+
/*
* LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
* 'soft reset' for domains and there is 'soft_reset' shutdown reason
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 28cea1f643..28ba9eb787 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -222,6 +222,20 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
config->arch.sve_vl = d_config->b_info.arch_arm.sve_vl / 128U;
}
+ switch (d_config->b_info.arch_arm.arm_sci.type) {
+ case LIBXL_ARM_SCI_TYPE_NONE:
+ config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
+ break;
+ case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
+ config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
+ break;
+ default:
+ LOG(ERROR, "Unknown ARM_SCI type %d",
+ d_config->b_info.arch_arm.arm_sci.type);
+ return ERROR_FAIL;
+ }
+ LOG(DEBUG, " - SCI type=%u", config->arch.arm_sci_type);
+
return 0;
}
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 33c9cfc1a2..aa2190ab5b 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -551,6 +551,15 @@ libxl_sve_type = Enumeration("sve_type", [
(2048, "2048")
], init_val = "LIBXL_SVE_TYPE_DISABLED")
+libxl_arm_sci_type = Enumeration("arm_sci_type", [
+ (0, "none"),
+ (1, "scmi_smc")
+ ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
+
+libxl_arm_sci = Struct("arm_sci", [
+ ("type", libxl_arm_sci_type),
+ ])
+
libxl_rdm_reserve = Struct("rdm_reserve", [
("strategy", libxl_rdm_reserve_strategy),
("policy", libxl_rdm_reserve_policy),
@@ -725,6 +734,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
("vuart", libxl_vuart_type),
("sve_vl", libxl_sve_type),
("nr_spis", uint32),
+ ("arm_sci", libxl_arm_sci),
])),
("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9a3679c023..bd22be9d33 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1284,6 +1284,36 @@ out:
if (rc) exit(EXIT_FAILURE);
}
+static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
+ const char *str)
+{
+ int ret = 0;
+ char *buf2, *ptr;
+ char *oparg;
+
+ if (NULL == (buf2 = ptr = strdup(str)))
+ return ERROR_NOMEM;
+
+ ptr = strtok(buf2, ",");
+ while (ptr != NULL)
+ {
+ if (MATCH_OPTION("type", ptr, oparg)) {
+ ret = libxl_arm_sci_type_from_string(oparg, &arm_sci->type);
+ if (ret) {
+ fprintf(stderr, "Unknown ARM_SCI type: %s\n", oparg);
+ ret = ERROR_INVAL;
+ goto parse_error;
+ }
+ }
+
+ ptr = strtok(NULL, ",");
+ }
+
+parse_error:
+ free(buf2);
+ return ret;
+}
+
void parse_config_data(const char *config_source,
const char *config_data,
int config_len,
@@ -2981,6 +3011,12 @@ skip_usbdev:
if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
b_info->arch_arm.nr_spis = l;
+ if (!xlu_cfg_get_string(config, "arm_sci", &buf, 1)) {
+ if (parse_arm_sci_config(config, &b_info->arch_arm.arm_sci, buf)) {
+ exit(EXIT_FAILURE);
+ }
+ }
+
parse_vkb_list(config, d_config);
d_config->virtios = NULL;
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index a09c4c4bd7..0a00f03a25 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -815,6 +815,36 @@ static int __init construct_domU(struct domain *d,
return rc;
}
+int __init domu_dt_sci_parse(struct dt_device_node *node,
+ struct xen_domctl_createdomain *d_cfg)
+{
+ const char *sci_type = NULL;
+ int ret;
+
+ d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
+
+ if ( !IS_ENABLED(CONFIG_ARM_SCI) ||
+ !dt_property_read_bool(node, "xen,sci_type") )
+ return 0;
+
+ ret = dt_property_read_string(node, "xen,sci_type", &sci_type);
+ if ( ret )
+ return ret;
+
+ if ( !strcmp(sci_type, "none") )
+ d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
+ else if ( !strcmp(sci_type, "scmi_smc") )
+ d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
+ else
+ {
+ printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
+ sci_type, dt_node_name(node));
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
void __init create_domUs(void)
{
struct dt_device_node *node;
@@ -975,7 +1005,8 @@ void __init create_domUs(void)
if ( !llc_coloring_enabled && llc_colors_str )
panic("'llc-colors' found, but LLC coloring is disabled\n");
- d_cfg.arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
+ if ( domu_dt_sci_parse(node, &d_cfg) )
+ panic("Error getting SCI configuration\n");
/*
* The variable max_init_domid is initialized with zero, so here it's
diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
index bbf88fbb9a..5c5f0880c4 100644
--- a/xen/arch/arm/firmware/Kconfig
+++ b/xen/arch/arm/firmware/Kconfig
@@ -25,7 +25,9 @@ config SCMI_SMC
doorbell mechanism and Shared Memory for transport ("arm,scmi-smc"
compatible only). The value of "arm,smc-id" DT property from SCMI
firmware node is used to trap and forward corresponding SCMI SMCs
- to firmware running at EL3, for calls coming from the hardware domain.
+ to firmware running at EL3, for calls coming from the hardware domain or
+ driver domain.
+ Use with EL3 firmware which supports only single SCMI OSPM agent.
endchoice
diff --git a/xen/arch/arm/firmware/scmi-smc.c b/xen/arch/arm/firmware/scmi-smc.c
index 13d1137592..7470a21505 100644
--- a/xen/arch/arm/firmware/scmi-smc.c
+++ b/xen/arch/arm/firmware/scmi-smc.c
@@ -14,6 +14,8 @@
#include <xen/device_tree.h>
#include <xen/errno.h>
#include <xen/init.h>
+#include <xen/iocap.h>
+#include <xen/param.h>
#include <xen/sched.h>
#include <xen/types.h>
@@ -22,7 +24,11 @@
#define SCMI_SMC_ID_PROP "arm,smc-id"
+static bool __ro_after_init opt_dom0_scmi_smc_passthrough = false;
+boolean_param("dom0_scmi_smc_passthrough", opt_dom0_scmi_smc_passthrough);
+
static uint32_t __ro_after_init scmi_smc_id;
+static struct domain __read_mostly *scmi_dom;
/*
* Check if provided SMC Function Identifier matches the one known by the SCMI
@@ -50,7 +56,7 @@ static bool scmi_handle_smc(struct cpu_user_regs *regs)
return false;
/* Only the hardware domain should use SCMI calls */
- if ( !is_hardware_domain(current->domain) )
+ if ( scmi_dom != current->domain )
{
gdprintk(XENLOG_WARNING, "SCMI: Unprivileged access attempt\n");
return false;
@@ -75,12 +81,45 @@ static bool scmi_handle_smc(struct cpu_user_regs *regs)
return true;
}
+static int
+scmi_smc_domain_sanitise_config(struct xen_domctl_createdomain *config)
+{
+ if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
+ config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC )
+ return -EINVAL;
+
+ return 0;
+}
+
static int scmi_smc_domain_init(struct domain *d,
struct xen_domctl_createdomain *config)
{
- if ( !is_hardware_domain(d) )
+ /*
+ * scmi_passthrough is not enabled:
+ * - proceed only for hw_domain
+ * - fail if guest domain has SCMI enabled.
+ */
+ if ( !opt_dom0_scmi_smc_passthrough && !is_hardware_domain(d) )
+ {
+ if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC )
+ return -EINVAL;
+ else
+ return 0;
+ }
+ /*
+ * scmi_passthrough is enabled:
+ * - ignore hw_domain
+ * - proceed only for domain with SCMI enabled.
+ */
+ if ( opt_dom0_scmi_smc_passthrough &&
+ (config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE ||
+ is_hardware_domain(d)) )
return 0;
+ if ( scmi_dom )
+ return -EEXIST;
+
+ scmi_dom = d;
d->arch.sci_enabled = true;
printk(XENLOG_DEBUG "SCMI: %pd init\n", d);
return 0;
@@ -88,12 +127,77 @@ static int scmi_smc_domain_init(struct domain *d,
static void scmi_smc_domain_destroy(struct domain *d)
{
- if ( !is_hardware_domain(d) )
+ if ( scmi_dom && scmi_dom != d )
return;
+ scmi_dom = NULL;
+ d->arch.sci_enabled = false;
printk(XENLOG_DEBUG "SCMI: %pd destroy\n", d);
}
+/*
+ * Handle Dom0 SCMI SMC specific DT nodes
+ *
+ * if dom0_scmi_smc_passthrough=false:
+ * - Copy SCMI nodes into Dom0 device tree.
+ * if dom0_scmi_smc_passthrough=true:
+ * - skip SCMI nodes from Dom0 DT
+ * - give dom0 control access to SCMI shmem MMIO, so SCMI can be passed
+ * through to guest.
+ */
+static bool scmi_smc_dt_handle_node(struct domain *d,
+ struct dt_device_node *node)
+{
+ static const struct dt_device_match shmem_matches[] __initconst = {
+ DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
+ { /* sentinel */ },
+ };
+ static const struct dt_device_match scmi_matches[] __initconst = {
+ DT_MATCH_PATH("/firmware/scmi"),
+ { /* sentinel */ },
+ };
+
+ /* skip scmi shmem node for dom0 if scmi not enabled */
+ if ( dt_match_node(shmem_matches, node) && !sci_domain_is_enabled(d) )
+ {
+ dt_dprintk("Skip scmi shmem node\n");
+ return true;
+ }
+
+ /*
+ * skip scmi node for dom0 if scmi not enabled, but give dom0 control
+ * access to SCMI shmem
+ */
+ if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
+ {
+ struct dt_device_node *shmem_node;
+ const __be32 *prop;
+ u64 paddr, size;
+ int ret;
+
+ /* give dom0 control access to SCMI shmem */
+ prop = dt_get_property(node, "shmem", NULL);
+ if ( !prop )
+ return true;
+
+ shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
+ if ( !shmem_node )
+ return true;
+
+ ret = dt_device_get_address(shmem_node, 0, &paddr, &size);
+ if ( ret )
+ return true;
+
+ ret = iomem_permit_access(d, paddr_to_pfn(paddr),
+ paddr_to_pfn(paddr + size - 1));
+
+ dt_dprintk("Skip scmi node\n");
+ return true;
+ }
+
+ return false;
+}
+
static int __init scmi_check_smccc_ver(void)
{
if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
@@ -108,8 +212,10 @@ static int __init scmi_check_smccc_ver(void)
static const struct sci_mediator_ops scmi_smc_ops = {
.handle_call = scmi_handle_smc,
+ .domain_sanitise_config = scmi_smc_domain_sanitise_config,
.domain_init = scmi_smc_domain_init,
.domain_destroy = scmi_smc_domain_destroy,
+ .dom0_dt_handle_node = scmi_smc_dt_handle_node,
};
/* Initialize the SCMI layer based on SMCs and Device-tree */
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH v4 4/8] docs: arm: add docs for SCMI over SMC calls forwarding driver
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
` (4 preceding siblings ...)
2025-05-19 15:50 ` [RFC PATCH v4 7/8] docs: arm: add SCI SCMI SMC multi-agent driver docs Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver Oleksii Moisieiev
` (2 subsequent siblings)
8 siblings, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
From: Grygorii Strashko <grygorii_strashko@epam.com>
Add documentation section for Simple Arm SCMI over SMC/HVC calls forwarding
driver (EL3).
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
---
.../arm/firmware/arm-scmi.rst | 177 ++++++++++++++++++
docs/hypervisor-guide/arm/index.rst | 9 +
docs/hypervisor-guide/index.rst | 1 +
3 files changed, 187 insertions(+)
create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi.rst
create mode 100644 docs/hypervisor-guide/arm/index.rst
diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi.rst
new file mode 100644
index 0000000000..bf6a458a6a
--- /dev/null
+++ b/docs/hypervisor-guide/arm/firmware/arm-scmi.rst
@@ -0,0 +1,177 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+ARM System Control and Management Interface (SCMI)
+==================================================
+
+The System Control and Management Interface (SCMI) [1], which is a set of operating
+system-independent software interfaces that are used in system management. SCMI currently
+provides interfaces for:
+
+- Discovery and self-description of the interfaces it supports
+- Power domain management
+- Clock management
+- Reset domain management
+- Voltage domain management
+- Sensor management
+- Performance management
+- Power capping and monitoring
+- Pin control protocol.
+
+The SCMI compliant firmware could run:
+
+- as part of EL3 secure world software (like Trusted Firmware-A) with
+ ARM SMC/HVC shared-memory transport;
+- on dedicated System Control Processor (SCP) with HW mailbox shared-memory transport
+
+The major purpose of enabling SCMI support in Xen is to enable guest domains access to the SCMI
+interfaces for performing management actions on passed-through devices (such as clocks/resets etc)
+without accessing directly to the System control HW (like clock controllers) which in most cases
+can't shared/split between domains. Or, at minimum, allow SCMI access for dom0/hwdom (or guest
+domain serving as Driver domain).
+
+The below sections describe SCMI support options available for Xen.
+
+[1] `Arm SCMI <https://developer.arm.com/documentation/den0056/latest/>`_
+
+Simple SCMI over SMC/HVC calls forwarding driver (EL3)
+------------------------------------------------------
+
+The EL3 SCMI firmware (TF-A) with a single SCMI OSPM agent support is pretty generic case for
+the default vendors SDK and new platforms with SCMI support. Such EL3 SCMI firmware supports only
+single SCMI OSPM transport (agent) with Shared memory based transport and SMC/HVC calls as doorbell.
+
+The SCMI over SMC/HVC calls forwarding driver solves major problem for this case by allowing
+SMC/HVC calls to be forwarded form guest to the EL3 SCMI firmware.
+
+By default, the SCMI over SMC/HVC calls forwarding is enabled for Dom0/hwdom.
+
+::
+
+ +--------------------------+
+ | |
+ | EL3 SCMI FW (TF-A) |
+ ++-------+--^--------------+
+ |shmem | | smc-id
+ +----^--+ |
+ | |
+ +----|-+---+---+----------+
+ | | | FWD | Xen |
+ | | +---^---+ |
+ +----|-----|--------------+
+ | | smc-id
+ +----v-----+--+ +---------+
+ | | | |
+ | Dom0/hwdom | | DomU |
+ | | | |
+ | | | |
+ +-------------+ +---------+
+
+
+The SCMI messages are passed directly through SCMI shared-memory (zero-copy) and driver only
+forwards SMC/HVC calls.
+
+Compiling
+^^^^^^^^^
+
+To build with the SCMI over SMC/HVC calls forwarding enabled support, enable Kconfig option
+
+::
+
+ CONFIG_SCMI_SMC
+
+The ``CONFIG_SCMI_SMC`` is enabled by default.
+
+Pass-through SCMI SMC to domain which serves as Driver domain
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This section describes how to configure the SCMI over SMC/HVC calls forwarding driver to handle use
+case "thin Dom0 with guest domain, which serves as Driver domain". In this case HW need to be
+enabled in Driver domain and dom0 is performing only control functions (without accessing FW) and so,
+the SCMI need to be enabled in Driver domain.
+
+::
+
+ +--------------------------+
+ |EL3 SCMI FW (TF-A) |
+ | |
+ +-------------^--+-------+-+
+ smc-id| |shmem0 |
+ | +----^--+
+ +-------------++------+|----+
+ |Xen | FWD || |
+ | +--^----+| |
+ +----------------|-----|----+
+ smc-id | |
+ +-----------+ +--+-----v-----+
+ | | | |
+ | Dom0 | | Driver |
+ | Control | | domain |
+ | | | |
+ +-----------+ +--------------+
+
+The SCMI can be enabled for one and only one guest domain.
+
+First. configure Dom0 to enable SCMI pass-through using Xen Command Line
+**"dom0_scmi_smc_passthrough"** option. This will disable SCMI for Dom0/hwdom and SCMI nodes will
+be removed from Dom0/hwdom device tree.
+
+**Configure SCMI pass-through for guest domain with toolstack**
+
+* In domain's xl.cfg file add **"arm_sci"** option as below
+
+::
+
+ arm_sci = "type=scmi_smc"
+
+* In domain's xl.cfg file enable access to the "arm,scmi-shmem"
+
+::
+
+ iomem = [
+ "47ff0,1@22001",
+ ]
+
+.. note:: It's up to the user to select guest IPA for mapping SCMI shared-memory.
+
+* Add SCMI nodes to the Driver domain partial device tree as in the below example:
+
+.. code::
+
+ passthrough {
+ scmi_shm_0: sram@22001000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x22001000 0x0 0x1000>;
+ };
+
+ firmware {
+ compatible = "simple-bus";
+ scmi: scmi {
+ compatible = "arm,scmi-smc";
+ shmem = <&scmi_shm_0>;
+ ...
+ }
+ }
+ }
+
+In general, the configuration is similar to any other HW pass-through, except explicitly
+enabling SCMI with "arm_sci" xl.cfg option.
+
+**Configure SCMI pass-through for predefined domain (dom0less)**
+
+* add "xen,sci_type" property for required DomU ("xen,domain") node
+
+::
+
+ xen,sci_type="scmi_smc"
+
+* add scmi nodes to the Driver domain partial device tree the same way as above and enable access
+ to the "arm,scmi-shmem" according to dom0less documentation. For example:
+
+.. code::
+
+ scmi_shm_0: sram@22001000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x00 0x22001000 0x00 0x1000>;
+ -> xen,reg = <0x0 0x47ff0000 0x0 0x1000 0x0 0x22001000>;
+ -> xen,force-assign-without-iommu;
+ };
diff --git a/docs/hypervisor-guide/arm/index.rst b/docs/hypervisor-guide/arm/index.rst
new file mode 100644
index 0000000000..7aae4a0a03
--- /dev/null
+++ b/docs/hypervisor-guide/arm/index.rst
@@ -0,0 +1,9 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+ARM
+===
+
+.. toctree::
+ :maxdepth: 2
+
+ firmware/arm-scmi
diff --git a/docs/hypervisor-guide/index.rst b/docs/hypervisor-guide/index.rst
index e4393b0697..520fe01554 100644
--- a/docs/hypervisor-guide/index.rst
+++ b/docs/hypervisor-guide/index.rst
@@ -9,3 +9,4 @@ Hypervisor documentation
code-coverage
x86/index
+ arm/index
\ No newline at end of file
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH v4 7/8] docs: arm: add SCI SCMI SMC multi-agent driver docs
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
` (3 preceding siblings ...)
2025-05-19 15:50 ` [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 4/8] docs: arm: add docs for SCMI over SMC calls forwarding driver Oleksii Moisieiev
` (3 subsequent siblings)
8 siblings, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
From: Grygorii Strashko <grygorii_strashko@epam.com>
Add SCI SCMI SMC multi-agent driver documentation.
It includes a detailed description of the SCMI multi-agent driver.
This document explains the driver's functionality, configuration,
and the compilation process. The Xen SCMI multi-agent driver is
designed to provide SCMI access to system resources from different
domains.
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
---
.../arm/firmware/arm-scmi.rst | 267 +++++++++++++++++-
1 file changed, 266 insertions(+), 1 deletion(-)
diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi.rst
index bf6a458a6a..27739015d5 100644
--- a/docs/hypervisor-guide/arm/firmware/arm-scmi.rst
+++ b/docs/hypervisor-guide/arm/firmware/arm-scmi.rst
@@ -31,7 +31,10 @@ domain serving as Driver domain).
The below sections describe SCMI support options available for Xen.
-[1] `Arm SCMI <https://developer.arm.com/documentation/den0056/latest/>`_
+| [1] `Arm SCMI <https://developer.arm.com/documentation/den0056/latest/>`_
+| [2] `System Control and Management Interface (SCMI) bindings <https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml>`_
+| [3] `Generic Domain Access Controllers bindings <https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml>`_
+
Simple SCMI over SMC/HVC calls forwarding driver (EL3)
------------------------------------------------------
@@ -175,3 +178,265 @@ enabling SCMI with "arm_sci" xl.cfg option.
-> xen,reg = <0x0 0x47ff0000 0x0 0x1000 0x0 0x22001000>;
-> xen,force-assign-without-iommu;
};
+
+SCMI SMC/HVC multi-agent driver (EL3)
+-------------------------------------
+
+The SCMI SMC/HVC multi-agent driver enables support for ARM EL3 Trusted Firmware-A (TF-A) which
+provides SCMI interface with multi-agnet support, as shown below.
+
+::
+
+ +-----------------------------------------+
+ | |
+ | EL3 TF-A SCMI |
+ +-------+--+-------+--+-------+--+-------++
+ |shmem0 | |shmem1 | |shmem2 | |shmemX |
+ +-----+-+ +---+---+ +--+----+ +---+---+
+ smc-id0 | | | |
+ agent0 | | | |
+ +-----v--------+---------+-----------+----+
+ | | | | |
+ | | | | |
+ +--------------+---------+-----------+----+
+ smc-id1 | smc-id2| smc-idX|
+ agent1 | agent2 | agentX |
+ | | |
+ +----v---+ +--v-----+ +--v-----+
+ | | | | | |
+ | Dom0 | | Dom1 | | DomX |
+ | | | | | |
+ | | | | | |
+ +--------+ +--------+ +--------+
+
+The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared-memory transport
+for every Agent in the system. The SCMI Agent transport channel defined by pair:
+
+- smc-id: SMC/HVC function id used for Doorbell
+- shmem: shared memory for messages transfer, **Xen page aligned** with mapping``p2m_mmio_direct_nc``.
+
+The following SCMI Agents expected to be defined by SCMI FW to enable SCMI multi-agent functionality
+under Xen:
+
+- Xen management agent: trusted agents that accesses to the Base Protocol commands to configure
+ agent specific permissions
+- OSPM VM agents: non-trusted agent, one for each Guest domain which is allowed direct HW access.
+ At least one OSPM VM agent has to be provided by FW if HW is handled only by Dom0 or Driver Domain.
+
+The EL3 SCMI FW expected to implement following Base protocol messages:
+
+- BASE_DISCOVER_AGENT
+- BASE_RESET_AGENT_CONFIGURATION (optional)
+- BASE_SET_DEVICE_PERMISSIONS (optional)
+
+The number of supported SCMI agents and their transport specifications are SCMI FW implementation
+specific.
+
+
+Compiling
+^^^^^^^^^
+
+To build with the SCMI SMC/HVC multi-agent driver support, enable Kconfig option:
+
+::
+
+ CONFIG_SCMI_SMC_MA
+
+
+Driver functionality
+^^^^^^^^^^^^^^^^^^^^
+
+The SCI SCMI SMC multi-agent driver implements following functionality:
+
+- The driver is initialized based on the Host DT SCMI node (only one SCMI interface is supported)
+ which describes Xen management agent SCMI interface.
+
+.. code::
+
+ scmi_shm_0 : sram@47ff0000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff0000 0x0 0x1000>;
+ };
+ firmware {
+ scmi: scmi {
+ compatible = "arm,scmi-smc";
+ arm, smc - id = <0x82000002>; <--- Xen manegement agent smc-id
+ \#address-cells = < 1>;
+ \#size-cells = < 0>;
+ \#access-controller - cells = < 1>;
+ shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
+
+ protocol@X{
+ };
+ };
+ };
+
+- The driver obtains Xen specific SCMI Agent's configuration from the Host DT, probes Agents and
+ builds SCMI Agents list. The Agents configuration is taken from "xen,scmi-secondary-agents"
+ property where first item is SCMI "agent_id", second - "arm,smc-id", and
+ third - "arm,scmi-shmem" phandle for this "agent_id":
+
+.. code::
+
+ chosen {
+ xen,scmi-secondary-agents = <
+ 1 0x82000003 &scmi_shm_1
+ 2 0x82000004 &scmi_shm_2
+ 3 0x82000005 &scmi_shm_3
+ 4 0x82000006 &scmi_shm_4>;
+ }
+
+ /{
+ scmi_shm_1: sram@47ff1000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff1000 0x0 0x1000>;
+ };
+ scmi_shm_2: sram@47ff2000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff2000 0x0 0x1000>;
+ };
+ scmi_shm_3: sram@47ff3000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff3000 0x0 0x1000>;
+ };
+ scmi_shm_4: sram@47ff4000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff4000 0x0 0x1000>;
+ };
+ }
+
+
+.. note::
+
+ Note that Xen is the only one entry in the system which need to know about SCMI multi-agent support.
+
+- The driver implements the SCI subsystem interface required for configuring and enabling SCMI
+ functionality for Dom0/hwdom and Guest domains. To enable SCMI functionality for guest domain
+ it has to be configured with unique supported SCMI Agent_id and use corresponding SCMI SMC/HVC
+ shared-memory transport ``[smc-id, shmem]`` defined for this SCMI Agent_id.
+
+- Once Xen domain is configured it can communicate with EL3 SCMI FW:
+
+ - zero-copy, the guest domain puts/gets SCMI message in/from shmem;
+ - the guest triggers SMC/HVC exception with agent "smc-id" (doorbell);
+ - the Xen driver catches exception, do checks and synchronously forwards it to EL3 FW.
+
+- the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen management agent channel on
+ domain destroy event. This allows to reset resources used by domain and so implement use-case
+ like domain reboot.
+
+
+Configure SCMI for Dom0
+^^^^^^^^^^^^^^^^^^^^^^^
+
+The **"dom0_scmi_agent_id=<dom0_agent_id>"** Xen command line is used to enable SCMI functionality for
+Dom0. if not provided SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
+
+The driver updates Dom0 DT SCMI node "arm,smc-id" property with value and fixes up "shmem" property
+according to the assigned for Dom0 SCMI "agent_id".
+
+Configure SCMI for for guest domain with toolstack
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* In domain's xl.cfg file add **"arm_sci"** option as below
+
+::
+
+ arm_sci = "type=scmi_smc_multiagent,agent_id=2"
+
+* In domain's xl.cfg file enable access to the "arm,scmi-shmem" which should correspond
+ assigned "agent_id" for the domain, for example:
+
+::
+
+ iomem = [
+ "47ff2,1@22001",
+ ]
+
+.. note:: It's up to the user to select guest IPA for mapping SCMI shared-memory.
+
+* Add SCMI nodes to the Driver domain partial device tree as in the below example.
+ The "arm,smc-id" should correspond assigned agent_id for the domain:
+
+.. code::
+
+ passthrough {
+ scmi_shm_0: sram@22001000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x22001000 0x0 0x1000>;
+ };
+
+ firmware {
+ compatible = "simple-bus";
+ scmi: scmi {
+ compatible = "arm,scmi-smc";
+ arm,smc-id = <0x82000004>;
+ shmem = <&scmi_shm_0>;
+ ...
+ }
+ }
+ }
+
+**Device specific access control**
+
+The XEN SCMI SMC/HVC multi-agent driver performs "access-controller" provider function in case
+EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
+BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
+The Host DT SCMI node should have "#access-controller-cells=<1>" property and DT devices should
+be bound to the SCMI node using Access Controllers bindings [3].
+
+For example:
+
+.. code::
+
+ &i2c1 {
+ access-controllers = <&scmi 0>;
+ };
+
+Use domain's xl.cfg file **"dtdev"** property to assign SCMI devices from toolstack to the guest:
+
+::
+
+ dtdev = [
+ "/soc/i2c@e6508000",
+ ]
+
+.. note::
+
+ xl.cfg:"dtdev" need contain all nodes which are under SCMI management (not only those which are
+ behind IOMMU) and passed-through to the guest domain.
+
+Configure SCMI for predefined domains (dom0less)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* add "xen,sci_type" and "xen,sci_agent_id" properties for required DomU ("xen,domain") node
+
+::
+
+ xen,sci_type="scmi_smc_multiagent"
+ xen,sci_agent_id=2
+
+* add scmi nodes to the Driver domain partial device tree the same way as above (toolstack case) and
+ enable access to the "arm,scmi-shmem" according to the dom0less documentation. For example:
+
+.. code::
+
+ scmi_shm_0: sram@22001000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x00 0x22001000 0x00 0x1000>;
+ -> xen,reg = <0x0 0x47ff0000 0x0 0x1000 0x0 0x22001000>;
+ -> xen,force-assign-without-iommu;
+ };
+
+* For SCMI device access control configure pass-through devices in the guest partial DT according to
+ the dom0less documentation and ensure that devices SCMI management has "xen,path" property set:
+
+.. code::
+
+ i2c@e6508000 {
+ ...
+ reg = <0x00 0xe6508000 0x00 0x1000>;
+ -> xen,path = "/soc/i2c@e6508000"
+ -> xen,reg = <0x0 0xe6508000 0x0 0x1000 0x0 0xe6508000>;
+ -> xen,force-assign-without-iommu;
+ };
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
` (2 preceding siblings ...)
2025-05-19 15:50 ` [RFC PATCH v4 3/8] xen/arm: scmi-smc: passthrough SCMI SMC to domain, single agent Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-19 18:54 ` Jan Beulich
2025-05-22 0:25 ` Stefano Stabellini
2025-05-19 15:50 ` [RFC PATCH v4 7/8] docs: arm: add SCI SCMI SMC multi-agent driver docs Oleksii Moisieiev
` (4 subsequent siblings)
8 siblings, 2 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
From: Grygorii Strashko <grygorii_strashko@epam.com>
Add chained handling of assigned DT devices to support access-controller
functionality through SCI framework, so DT device assign request can be
passed to FW for processing and enabling VM access to requested device
(for example, device power management through FW interface like SCMI).
The SCI access-controller DT device processing is chained after IOMMU
processing and expected to be executed for any DT device regardless of its
protection by IOMMU (or if IOMMU is disabled).
This allows to pass not only IOMMU protected DT device through
xl.cfg:"dtdev" property for processing:
dtdev = [
"/soc/video@e6ef0000", <- IOMMU protected device
"/soc/i2c@e6508000", <- not IOMMU protected device
]
The change is done in two parts:
1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
not fail if DT device is not protected by IOMMU
2) add chained call to sci_do_domctl() in do_domctl()
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
---
xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
xen/common/domctl.c | 19 +++++++++++++
xen/drivers/passthrough/device_tree.c | 6 ++++
4 files changed, 76 insertions(+)
diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
index e1522e10e2..8efd541c4f 100644
--- a/xen/arch/arm/firmware/sci.c
+++ b/xen/arch/arm/firmware/sci.c
@@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
return 0;
}
+int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
+ XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+ struct dt_device_node *dev;
+ int ret = 0;
+
+ switch ( domctl->cmd )
+ {
+ case XEN_DOMCTL_assign_device:
+ ret = -EOPNOTSUPP;
+ if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT )
+ break;
+
+ if ( !cur_mediator )
+ break;
+
+ if ( !cur_mediator->assign_dt_device )
+ break;
+
+ ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
+ domctl->u.assign_device.u.dt.size, &dev);
+ if ( ret )
+ return ret;
+
+ ret = sci_assign_dt_device(d, dev);
+ if ( ret )
+ break;
+
+ break;
+ default:
+ /* do not fail here as call is chained with iommu handling */
+ break;
+ }
+
+ return ret;
+}
+
static int __init sci_init(void)
{
struct dt_device_node *np;
diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
index 71fb54852e..b8d1bc8a62 100644
--- a/xen/arch/arm/include/asm/firmware/sci.h
+++ b/xen/arch/arm/include/asm/firmware/sci.h
@@ -146,6 +146,14 @@ int sci_dt_finalize(struct domain *d, void *fdt);
* control" functionality.
*/
int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
+
+/*
+ * SCI domctl handler
+ *
+ * Only XEN_DOMCTL_assign_device is handled for now.
+ */
+int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
+ XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
#else
static inline bool sci_domain_is_enabled(struct domain *d)
@@ -195,6 +203,12 @@ static inline int sci_assign_dt_device(struct domain *d,
return 0;
}
+static inline int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
+ XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+ return 0;
+}
+
#endif /* CONFIG_ARM_SCI */
#endif /* __ASM_ARM_SCI_H */
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 05abb581a0..a74ee92067 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -27,6 +27,7 @@
#include <xen/vm_event.h>
#include <xen/monitor.h>
#include <asm/current.h>
+#include <asm/firmware/sci.h>
#include <asm/irq.h>
#include <asm/page.h>
#include <asm/p2m.h>
@@ -851,6 +852,24 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
case XEN_DOMCTL_deassign_device:
case XEN_DOMCTL_get_device_group:
ret = iommu_do_domctl(op, d, u_domctl);
+
+ if ( !ret || ret == -EOPNOTSUPP )
+ {
+ int ret1;
+ /*
+ * Add chained handling of assigned DT devices to support
+ * access-controller functionality through SCI framework, so
+ * DT device assign request can be passed to FW for processing and
+ * enabling VM access to requested device.
+ * The access-controller DT device processing is chained after IOMMU
+ * processing and expected to be executed for any DT device
+ * regardless if DT device is protected by IOMMU or not (or IOMMU
+ * is disabled).
+ */
+ ret1 = sci_do_domctl(op, d, u_domctl);
+ if ( ret1 != -EOPNOTSUPP )
+ ret = ret1;
+ }
break;
case XEN_DOMCTL_get_paging_mempool_size:
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 075fb25a37..2624767e51 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -318,6 +318,12 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
break;
}
+ if ( !dt_device_is_protected(dev) )
+ {
+ ret = 0;
+ break;
+ }
+
ret = iommu_assign_dt_device(d, dev);
if ( ret )
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
` (5 preceding siblings ...)
2025-05-19 15:50 ` [RFC PATCH v4 4/8] docs: arm: add docs for SCMI over SMC calls forwarding driver Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-23 20:06 ` Stefano Stabellini
2025-06-18 7:28 ` Julien Grall
2025-05-19 15:50 ` [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent Oleksii Moisieiev
2025-06-05 22:40 ` [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Julien Grall
8 siblings, 2 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
(TF-A) which provides SCMI interface with multi-agnet support, as shown
below.
+-----------------------------------------+
| |
| EL3 TF-A SCMI |
+-------+--+-------+--+-------+--+-------++
|shmem0 | |shmem1 | |shmem2 | |shmemX |
+-----+-+ +---+---+ +--+----+ +---+---+
smc-id0 | | | |
agent0 | | | |
+-----v--------+---------+-----------+----+
| | | | |
| | | | |
+--------------+---------+-----------+----+
smc-id1 | smc-id2| smc-idX|
agent1 | agent2 | agentX |
| | |
+----v---+ +--v-----+ +--v-----+
| | | | | |
| Dom0 | | Dom1 | | DomX |
| | | | | |
| | | | | |
+--------+ +--------+ +--------+
The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
memory transport for every Agent in the system.
The SCMI Agent transport channel defined by pair:
- smc-id: SMC/HVC id used for Doorbell
- shmem: shared memory for messages transfer, Xen page aligned,
p2m_mmio_direct_nc.
The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
multi-agent functionality under Xen:
- Xen manegement agent: trusted agents that accesses to the Base Protocol
commands to configure agent specific permissions
- OSPM VM agents: non-trusted agent, one for each Guest domain which is
allowed direct HW access. At least one OSPM VM agent has to be provided
by FW if HW is handled only by Dom0 or Driver Domain.
The EL3 SCMI FW expected to implement following Base protocol messages:
- BASE_DISCOVER_AGENT
- BASE_RESET_AGENT_CONFIGURATION (optional)
- BASE_SET_DEVICE_PERMISSIONS (optional)
The SCI SCMI SMC multi-agent driver implements following functionality:
- It's initialized based on the Host DT SCMI node (only one SCMI interface
is supported) which describes Xen management agent SCMI interface.
scmi_shm_0 : sram@47ff0000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff0000 0x0 0x1000>;
};
firmware {
scmi: scmi {
compatible = "arm,scmi-smc";
arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
\#address-cells = < 1>;
\#size-cells = < 0>;
\#access-controller - cells = < 1>;
shmem = <&scmi_shm_0>; // Xen manegement agent shmem
protocol@X{
};
};
};
- It obtains Xen specific SCMI Agent's configuration from the Host DT,
probes Agents and build SCMI Agents list; The Agents configuration is taken from:
chosen {
xen,scmi-secondary-agents = <
1 0x82000003 &scmi_shm_1
2 0x82000004 &scmi_shm_2
3 0x82000005 &scmi_shm_3
4 0x82000006 &scmi_shm_4>;
}
/{
scmi_shm_1: sram@47ff1000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff1000 0x0 0x1000>;
};
scmi_shm_2: sram@47ff2000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff2000 0x0 0x1000>;
};
scmi_shm_3: sram@47ff3000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff3000 0x0 0x1000>;
};
}
where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
this agent_id.
Note that Xen is the only one entry in the system which need to know
about SCMI multi-agent support.
- It implements the SCI subsystem interface required for configuring and
enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
SCMI functionality for domain it has to be configured with unique supported
SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
[smc-id, shmem] defined for this SCMI Agent_id.
- Once Xen domain is configured it can communicate with EL3 SCMI FW:
-- zero-copy, the guest domain puts SCMI message in shmem;
-- the guest triggers SMC/HVC exception with smc-id (doorbell);
-- the Xen driver catches exception, do checks and synchronously forwards
it to EL3 FW.
- the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
management agent channel on domain destroy event. This allows to reset
resources used by domain and so implement use-case like domain reboot.
Dom0 Enable SCMI SMC:
- pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
node according to assigned agent_id.
Guest domains enable SCMI SMC:
- xl.cfg: add configuration option as below
arm_sci = "type=scmi_smc_multiagent,agent_id=2"
- xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
the domain, for example:
iomem = [
"47ff2,1@22001",
]
- DT: add SCMI nodes to the Driver domain partial device tree as in the
below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
passthrough {
scmi_shm_0: sram@22001000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x22001000 0x0 0x1000>;
};
firmware {
compatible = "simple-bus";
scmi: scmi {
compatible = "arm,scmi-smc";
arm,smc-id = <0x82000004>;
shmem = <&scmi_shm_0>;
...
}
}
}
SCMI "4.2.1.1 Device specific access control"
The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
to the Xen SCMI.
&i2c1 {
access-controllers = <&scmi 0>;
};
The Dom0 and dom0less domains DT devices will be processed automatically through
sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
shell be used:
dtdev = [
"/soc/i2c@e6508000",
]
xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
[1] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
[2] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
Changes in v4:
- toolstack comments from Anthony PERARD
- added dom0less support
- added doc for "xen,scmi-secondary-agents"
docs/man/xl.cfg.5.pod.in | 13 +
docs/misc/arm/device-tree/booting.txt | 60 ++
docs/misc/xen-command-line.pandoc | 9 +
tools/libs/light/libxl_arm.c | 4 +
tools/libs/light/libxl_types.idl | 4 +-
tools/xl/xl_parse.c | 12 +
xen/arch/arm/dom0less-build.c | 11 +
xen/arch/arm/domain_build.c | 3 +-
xen/arch/arm/firmware/Kconfig | 11 +
xen/arch/arm/firmware/Makefile | 1 +
xen/arch/arm/firmware/scmi-proto.h | 164 ++++
xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
xen/arch/arm/firmware/scmi-shmem.h | 45 +
xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
xen/include/public/arch-arm.h | 3 +
15 files changed, 1371 insertions(+), 2 deletions(-)
create mode 100644 xen/arch/arm/firmware/scmi-proto.h
create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 1ccf50b8ea..302c46d8bc 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
option.
+=item B<scmi_smc_multiagent>
+
+Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
+SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
+with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
+specified for the guest.
+
=back
+=item B<agent_id=NUMBER>
+
+Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
+if the SCMI SMC support is enabled for the guest. The agent ids of domains
+existing on a single host must be unique and in the range [1..255].
+
=back
=back
diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 8943c04173..c8923ab8b2 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -296,6 +296,20 @@ with the following properties:
Should be used together with dom0_scmi_smc_passthrough Xen command line
option.
+ - "scmi_smc_multiagent"
+
+ Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
+ SMC calls forwarding from domain to the EL3 firmware (like ARM
+ Trusted Firmware-A) with a multi SCMI OSPM agent support.
+ The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
+ property.
+
+- "xen,sci_agent_id"
+
+ Specifies a non-zero ARM SCI agent id for the guest. This option is
+ mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
+ the guest. The agent ids of guest must be unique and in the range [1..255].
+
Under the "xen,domain" compatible node, one or more sub-nodes are present
for the DomU kernel and ramdisk.
@@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
guest physical address space. DomU1 is explicitly defined as the owner domain,
and DomU2 is the borrower domain.
+
+SCMI SMC multi-agent support
+============================
+
+For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
+the Xen specific SCMI Agent's configuration shell be provided in the Host DT
+according to the SCMI compliant EL3 Firmware specification with
+ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
+the top-level "chosen" node:
+
+- xen,scmi-secondary-agents
+
+ Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
+ available for Xen. Each Agent defined as triple consisting of:
+ SCMI agent_id,
+ SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
+ phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
+
+As an example:
+
+chosen {
+ xen,scmi-secondary-agents = <
+ 1 0x82000003 &scmi_shm_1
+ 2 0x82000004 &scmi_shm_2
+ 3 0x82000005 &scmi_shm_3
+ 4 0x82000006 &scmi_shm_4>;
+}
+
+/{
+ scmi_shm_1: sram@47ff1000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff1000 0x0 0x1000>;
+ };
+ scmi_shm_2: sram@47ff2000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff2000 0x0 0x1000>;
+ };
+ scmi_shm_3: sram@47ff3000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff3000 0x0 0x1000>;
+ };
+ scmi_shm_3: sram@47ff4000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff4000 0x0 0x1000>;
+ };
+}
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 8e50f6b7c7..bc3c64d6ec 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
SCMI nodes removed from Dom0/hwdom device tree.
(for example, thin Dom0 with Driver domain use-case).
+### dom0_scmi_agent_id (ARM)
+> `= <integer>`
+
+The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
+enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
+The SCMI will be disabled for Dom0 if this option is not specified
+(for example, thin Dom0 or dom0less use-cases).
+The agent ids of domains existing on a single host must be unique.
+
### dtuart (ARM)
> `= path [:options]`
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 28ba9eb787..7712f53cd4 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
break;
+ case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
+ config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
+ config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
+ break;
default:
LOG(ERROR, "Unknown ARM_SCI type %d",
d_config->b_info.arch_arm.arm_sci.type);
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index aa2190ab5b..11e31ce786 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
libxl_arm_sci_type = Enumeration("arm_sci_type", [
(0, "none"),
- (1, "scmi_smc")
+ (1, "scmi_smc"),
+ (2, "scmi_smc_multiagent")
], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
libxl_arm_sci = Struct("arm_sci", [
("type", libxl_arm_sci_type),
+ ("agent_id", uint8)
])
libxl_rdm_reserve = Struct("rdm_reserve", [
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index bd22be9d33..81aa3797e3 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
}
}
+ if (MATCH_OPTION("agent_id", ptr, oparg)) {
+ unsigned long val = parse_ulong(oparg);
+
+ if (!val || val > 255) {
+ fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
+ val);
+ ret = ERROR_INVAL;
+ goto parse_error;
+ }
+ arm_sci->agent_id = val;
+ }
+
ptr = strtok(NULL, ",");
}
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 0a00f03a25..43d21eb889 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
else if ( !strcmp(sci_type, "scmi_smc") )
d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
+ else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
+ {
+ uint32_t agent_id = 0;
+
+ if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
+ !agent_id )
+ return -EINVAL;
+
+ d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
+ d_cfg->arch.arm_sci_agent_id = agent_id;
+ }
else
{
printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 36d28b52a4..0c9274a2b3 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
- dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
+ dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
+ dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
continue;
if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") )
diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
index 5c5f0880c4..6b051c8ada 100644
--- a/xen/arch/arm/firmware/Kconfig
+++ b/xen/arch/arm/firmware/Kconfig
@@ -29,6 +29,17 @@ config SCMI_SMC
driver domain.
Use with EL3 firmware which supports only single SCMI OSPM agent.
+config SCMI_SMC_MA
+ bool "Enable ARM SCMI SMC multi-agent driver"
+ select ARM_SCI
+ help
+ Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
+ to EL3 firmware (TF-A) which supports multi-agent feature.
+ This feature allows to enable SCMI per Domain using unique SCMI agent_id,
+ so Domain is identified by EL3 firmware as an SCMI Agent and can access
+ allowed platform resources through dedicated SMC/HVC Shared memory based
+ transport.
+
endchoice
endmenu
diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
index 71bdefc24a..37927e690e 100644
--- a/xen/arch/arm/firmware/Makefile
+++ b/xen/arch/arm/firmware/Makefile
@@ -1,2 +1,3 @@
obj-$(CONFIG_ARM_SCI) += sci.o
obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
+obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
new file mode 100644
index 0000000000..3f4b9c5d6b
--- /dev/null
+++ b/xen/arch/arm/firmware/scmi-proto.h
@@ -0,0 +1,164 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Arm System Control and Management Interface definitions
+ * Version 3.0 (DEN0056C)
+ *
+ * Copyright (c) 2024 EPAM Systems
+ */
+
+#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
+#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
+
+#include <xen/stdint.h>
+
+#define SCMI_SHORT_NAME_MAX_SIZE 16
+
+/* SCMI status codes. See section 4.1.4 */
+#define SCMI_SUCCESS 0
+#define SCMI_NOT_SUPPORTED (-1)
+#define SCMI_INVALID_PARAMETERS (-2)
+#define SCMI_DENIED (-3)
+#define SCMI_NOT_FOUND (-4)
+#define SCMI_OUT_OF_RANGE (-5)
+#define SCMI_BUSY (-6)
+#define SCMI_COMMS_ERROR (-7)
+#define SCMI_GENERIC_ERROR (-8)
+#define SCMI_HARDWARE_ERROR (-9)
+#define SCMI_PROTOCOL_ERROR (-10)
+
+/* Protocol IDs */
+#define SCMI_BASE_PROTOCOL 0x10
+
+/* Base protocol message IDs */
+#define SCMI_BASE_PROTOCOL_VERSION 0x0
+#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
+#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
+#define SCMI_BASE_DISCOVER_AGENT 0x7
+#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
+#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
+
+typedef struct scmi_msg_header {
+ uint8_t id;
+ uint8_t type;
+ uint8_t protocol;
+ uint32_t status;
+} scmi_msg_header_t;
+
+/* Table 2 Message header format */
+#define SCMI_HDR_ID GENMASK(7, 0)
+#define SCMI_HDR_TYPE GENMASK(9, 8)
+#define SCMI_HDR_PROTO GENMASK(17, 10)
+
+#define SCMI_FIELD_GET(_mask, _reg) \
+ ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
+#define SCMI_FIELD_PREP(_mask, _val) \
+ (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
+
+static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
+{
+ return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
+ SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
+ SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
+}
+
+static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
+{
+ hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
+ hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
+ hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
+}
+
+static inline int scmi_to_xen_errno(int scmi_status)
+{
+ if ( scmi_status == SCMI_SUCCESS )
+ return 0;
+
+ switch ( scmi_status )
+ {
+ case SCMI_NOT_SUPPORTED:
+ return -EOPNOTSUPP;
+ case SCMI_INVALID_PARAMETERS:
+ return -EINVAL;
+ case SCMI_DENIED:
+ return -EACCES;
+ case SCMI_NOT_FOUND:
+ return -ENOENT;
+ case SCMI_OUT_OF_RANGE:
+ return -ERANGE;
+ case SCMI_BUSY:
+ return -EBUSY;
+ case SCMI_COMMS_ERROR:
+ return -ENOTCONN;
+ case SCMI_GENERIC_ERROR:
+ return -EIO;
+ case SCMI_HARDWARE_ERROR:
+ return -ENXIO;
+ case SCMI_PROTOCOL_ERROR:
+ return -EBADMSG;
+ default:
+ return -EINVAL;
+ }
+}
+
+/* PROTOCOL_VERSION */
+#define SCMI_VERSION_MINOR GENMASK(15, 0)
+#define SCMI_VERSION_MAJOR GENMASK(31, 16)
+
+struct scmi_msg_prot_version_p2a {
+ uint32_t version;
+} __packed;
+
+/* BASE PROTOCOL_ATTRIBUTES */
+#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
+#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
+
+struct scmi_msg_base_attributes_p2a {
+ uint32_t attributes;
+} __packed;
+
+/*
+ * BASE_DISCOVER_AGENT
+ */
+#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
+
+struct scmi_msg_base_discover_agent_a2p {
+ uint32_t agent_id;
+} __packed;
+
+struct scmi_msg_base_discover_agent_p2a {
+ uint32_t agent_id;
+ char name[SCMI_SHORT_NAME_MAX_SIZE];
+} __packed;
+
+/*
+ * BASE_SET_DEVICE_PERMISSIONS
+ */
+#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
+
+struct scmi_msg_base_set_device_permissions_a2p {
+ uint32_t agent_id;
+ uint32_t device_id;
+ uint32_t flags;
+} __packed;
+
+/*
+ * BASE_RESET_AGENT_CONFIGURATION
+ */
+#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
+
+struct scmi_msg_base_reset_agent_cfg_a2p {
+ uint32_t agent_id;
+ uint32_t flags;
+} __packed;
+
+#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
new file mode 100644
index 0000000000..dd613ee0b5
--- /dev/null
+++ b/xen/arch/arm/firmware/scmi-shmem.c
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
+ *
+ * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
+ * Copyright (c) 2025 EPAM Systems
+ */
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#include <asm/io.h>
+#include <xen/err.h>
+
+#include "scmi-proto.h"
+#include "scmi-shmem.h"
+
+/*
+ * Copy data from IO memory space to "real" memory space.
+ */
+static void __memcpy_fromio(void *to, const volatile void __iomem *from,
+ size_t count)
+{
+ while ( count && !IS_ALIGNED((unsigned long)from, 4) )
+ {
+ *(u8 *)to = readb_relaxed(from);
+ from++;
+ to++;
+ count--;
+ }
+
+ while ( count >= 4 )
+ {
+ *(u32 *)to = readl_relaxed(from);
+ from += 4;
+ to += 4;
+ count -= 4;
+ }
+
+ while ( count )
+ {
+ *(u8 *)to = readb_relaxed(from);
+ from++;
+ to++;
+ count--;
+ }
+}
+
+/*
+ * Copy data from "real" memory space to IO memory space.
+ */
+static void __memcpy_toio(volatile void __iomem *to, const void *from,
+ size_t count)
+{
+ while ( count && !IS_ALIGNED((unsigned long)to, 4) )
+ {
+ writeb_relaxed(*(u8 *)from, to);
+ from++;
+ to++;
+ count--;
+ }
+
+ while ( count >= 4 )
+ {
+ writel_relaxed(*(u32 *)from, to);
+ from += 4;
+ to += 4;
+ count -= 4;
+ }
+
+ while ( count )
+ {
+ writeb_relaxed(*(u8 *)from, to);
+ from++;
+ to++;
+ count--;
+ }
+}
+
+static inline int
+shmem_channel_is_free(const volatile struct scmi_shared_mem __iomem *shmem)
+{
+ return (readl(&shmem->channel_status) &
+ SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE) ? 0 : -EBUSY;
+}
+
+int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
+ scmi_msg_header_t *hdr, void *data, int len)
+{
+ int ret;
+
+ if ( (len + sizeof(shmem->msg_header)) > SCMI_SHMEM_MAPPED_SIZE )
+ {
+ printk(XENLOG_ERR "scmi: Wrong size of smc message. Data is invalid\n");
+ return -EINVAL;
+ }
+
+ ret = shmem_channel_is_free(shmem);
+ if ( ret )
+ return ret;
+
+ writel_relaxed(0x0, &shmem->channel_status);
+ /* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
+ writel_relaxed(0x0, &shmem->flags);
+ writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
+ writel(pack_scmi_header(hdr), &shmem->msg_header);
+
+ if ( len > 0 && data )
+ __memcpy_toio(shmem->msg_payload, data, len);
+
+ return 0;
+}
+
+int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
+ scmi_msg_header_t *hdr, void *data, int len)
+{
+ int recv_len;
+ int ret;
+ int pad = sizeof(hdr->status);
+
+ if ( len >= SCMI_SHMEM_MAPPED_SIZE - sizeof(shmem) )
+ {
+ printk(XENLOG_ERR
+ "scmi: Wrong size of input smc message. Data may be invalid\n");
+ return -EINVAL;
+ }
+
+ ret = shmem_channel_is_free(shmem);
+ if ( ret )
+ return ret;
+
+ recv_len = readl(&shmem->length) - sizeof(shmem->msg_header);
+
+ if ( recv_len < 0 )
+ {
+ printk(XENLOG_ERR
+ "scmi: Wrong size of smc message. Data may be invalid\n");
+ return -EINVAL;
+ }
+
+ unpack_scmi_header(readl(&shmem->msg_header), hdr);
+
+ hdr->status = readl(&shmem->msg_payload);
+ recv_len = recv_len > pad ? recv_len - pad : 0;
+
+ ret = scmi_to_xen_errno(hdr->status);
+ if ( ret )
+ {
+ printk(XENLOG_DEBUG "scmi: Error received: %d\n", ret);
+ return ret;
+ }
+
+ if ( recv_len > len )
+ {
+ printk(XENLOG_ERR
+ "scmi: Not enough buffer for message %d, expecting %d\n",
+ recv_len, len);
+ return -EINVAL;
+ }
+
+ if ( recv_len > 0 )
+ __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/firmware/scmi-shmem.h b/xen/arch/arm/firmware/scmi-shmem.h
new file mode 100644
index 0000000000..2f8e23ff76
--- /dev/null
+++ b/xen/arch/arm/firmware/scmi-shmem.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Arm System Control and Management Interface definitions
+ * Version 3.0 (DEN0056C)
+ * Shared Memory based Transport
+ *
+ * Copyright (c) 2024 EPAM Systems
+ */
+
+#ifndef XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
+#define XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
+
+#include <xen/stdint.h>
+
+#define SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE BIT(0, UL)
+#define SCMI_SHMEM_CHAN_STAT_CHANNEL_ERROR BIT(1, UL)
+
+struct scmi_shared_mem {
+ uint32_t reserved;
+ uint32_t channel_status;
+ uint32_t reserved1[2];
+ uint32_t flags;
+ uint32_t length;
+ uint32_t msg_header;
+ uint8_t msg_payload[];
+};
+
+#define SCMI_SHMEM_MAPPED_SIZE PAGE_SIZE
+
+int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
+ scmi_msg_header_t *hdr, void *data, int len);
+
+int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
+ scmi_msg_header_t *hdr, void *data, int len);
+#endif /* XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/firmware/scmi-smc-multiagent.c b/xen/arch/arm/firmware/scmi-smc-multiagent.c
new file mode 100644
index 0000000000..e023bca3a1
--- /dev/null
+++ b/xen/arch/arm/firmware/scmi-smc-multiagent.c
@@ -0,0 +1,860 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
+ *
+ * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
+ * Copyright (c) 2025 EPAM Systems
+ */
+
+#include <xen/acpi.h>
+
+#include <xen/device_tree.h>
+#include <xen/init.h>
+#include <xen/iocap.h>
+#include <xen/err.h>
+#include <xen/libfdt/libfdt.h>
+#include <xen/param.h>
+#include <xen/sched.h>
+#include <xen/vmap.h>
+
+#include <asm/firmware/sci.h>
+#include <asm/smccc.h>
+
+#include "scmi-proto.h"
+#include "scmi-shmem.h"
+
+#define SCMI_AGENT_ID_INVALID 0xFF
+
+static uint8_t __initdata opt_dom0_scmi_agent_id = SCMI_AGENT_ID_INVALID;
+integer_param("dom0_scmi_agent_id", opt_dom0_scmi_agent_id);
+
+#define SCMI_SECONDARY_AGENTS "xen,scmi-secondary-agents"
+
+#define HYP_CHANNEL 0x0
+
+struct scmi_channel {
+ uint32_t agent_id;
+ uint32_t func_id;
+ domid_t domain_id;
+ uint64_t paddr;
+ uint64_t len;
+ struct scmi_shared_mem __iomem *shmem;
+ spinlock_t lock;
+ struct list_head list;
+};
+
+struct scmi_data {
+ struct list_head channel_list;
+ spinlock_t channel_list_lock;
+ uint32_t func_id;
+ bool initialized;
+ uint32_t shmem_phandle;
+ struct dt_device_node *dt_dev;
+};
+
+static struct scmi_data scmi_data;
+
+static int send_smc_message(struct scmi_channel *chan_info,
+ scmi_msg_header_t *hdr, void *data, int len)
+{
+ struct arm_smccc_res resp;
+ int ret;
+
+ ret = shmem_put_message(chan_info->shmem, hdr, data, len);
+ if ( ret )
+ return ret;
+
+ arm_smccc_1_1_smc(chan_info->func_id, 0, 0, 0, 0, 0, 0, 0, &resp);
+
+ if ( resp.a0 )
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+static int do_smc_xfer(struct scmi_channel *chan_info, scmi_msg_header_t *hdr,
+ void *tx_data, int tx_size, void *rx_data, int rx_size)
+{
+ int ret = 0;
+
+ ASSERT(chan_info && chan_info->shmem);
+
+ if ( !hdr )
+ return -EINVAL;
+
+ spin_lock(&chan_info->lock);
+
+ printk(XENLOG_DEBUG
+ "scmi: agent_id = %d msg_id = %x type = %d, proto = %x\n",
+ chan_info->agent_id, hdr->id, hdr->type, hdr->protocol);
+
+ ret = send_smc_message(chan_info, hdr, tx_data, tx_size);
+ if ( ret )
+ goto clean;
+
+ ret = shmem_get_response(chan_info->shmem, hdr, rx_data, rx_size);
+
+clean:
+ printk(XENLOG_DEBUG
+ "scmi: get smc response agent_id = %d msg_id = %x proto = %x res=%d\n",
+ chan_info->agent_id, hdr->id, hdr->protocol, ret);
+
+ spin_unlock(&chan_info->lock);
+
+ return ret;
+}
+
+static struct scmi_channel *get_channel_by_id(uint32_t agent_id)
+{
+ struct scmi_channel *curr;
+ bool found = false;
+
+ spin_lock(&scmi_data.channel_list_lock);
+ list_for_each_entry(curr, &scmi_data.channel_list, list)
+ {
+ if ( curr->agent_id == agent_id )
+ {
+ found = true;
+ break;
+ }
+ }
+
+ spin_unlock(&scmi_data.channel_list_lock);
+ if ( found )
+ return curr;
+
+ return NULL;
+}
+
+static struct scmi_channel *acquire_scmi_channel(struct domain *d,
+ uint32_t agent_id)
+{
+ struct scmi_channel *curr;
+ struct scmi_channel *ret = ERR_PTR(-ENOENT);
+
+ spin_lock(&scmi_data.channel_list_lock);
+ list_for_each_entry(curr, &scmi_data.channel_list, list)
+ {
+ if ( curr->agent_id == agent_id )
+ {
+ if ( curr->domain_id != DOMID_INVALID )
+ {
+ ret = ERR_PTR(-EEXIST);
+ break;
+ }
+
+ curr->domain_id = d->domain_id;
+ ret = curr;
+ break;
+ }
+ }
+
+ spin_unlock(&scmi_data.channel_list_lock);
+
+ return ret;
+}
+
+static void relinquish_scmi_channel(struct scmi_channel *channel)
+{
+ ASSERT(channel != NULL);
+
+ spin_lock(&scmi_data.channel_list_lock);
+ channel->domain_id = DOMID_INVALID;
+ spin_unlock(&scmi_data.channel_list_lock);
+}
+
+static int map_channel_memory(struct scmi_channel *channel)
+{
+ ASSERT(channel && channel->paddr);
+ channel->shmem = ioremap_nocache(channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
+ if ( !channel->shmem )
+ return -ENOMEM;
+
+ channel->shmem->channel_status = SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE;
+ printk(XENLOG_DEBUG "scmi: Got shmem %lx after vmap %p\n", channel->paddr,
+ channel->shmem);
+
+ return 0;
+}
+
+static void unmap_channel_memory(struct scmi_channel *channel)
+{
+ ASSERT(channel && channel->shmem);
+ iounmap(channel->shmem);
+ channel->shmem = NULL;
+}
+
+static struct scmi_channel *smc_create_channel(uint32_t agent_id,
+ uint32_t func_id, uint64_t addr)
+{
+ struct scmi_channel *channel;
+
+ channel = get_channel_by_id(agent_id);
+ if ( channel )
+ return ERR_PTR(EEXIST);
+
+ channel = xmalloc(struct scmi_channel);
+ if ( !channel )
+ return ERR_PTR(ENOMEM);
+
+ spin_lock_init(&channel->lock);
+ channel->agent_id = agent_id;
+ channel->func_id = func_id;
+ channel->domain_id = DOMID_INVALID;
+ channel->shmem = NULL;
+ channel->paddr = addr;
+ list_add_tail(&channel->list, &scmi_data.channel_list);
+ return channel;
+}
+
+static void free_channel_list(void)
+{
+ struct scmi_channel *curr, *_curr;
+
+ list_for_each_entry_safe(curr, _curr, &scmi_data.channel_list, list)
+ {
+ list_del(&curr->list);
+ xfree(curr);
+ }
+}
+
+static int __init
+scmi_dt_read_hyp_channel_addr(struct dt_device_node *scmi_node, u64 *addr,
+ u64 *size)
+{
+ struct dt_device_node *shmem_node;
+ const __be32 *prop;
+
+ prop = dt_get_property(scmi_node, "shmem", NULL);
+ if ( !prop )
+ return -EINVAL;
+
+ shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
+ if ( IS_ERR_OR_NULL(shmem_node) )
+ {
+ printk(XENLOG_ERR
+ "scmi: Device tree error, can't parse reserved memory %ld\n",
+ PTR_ERR(shmem_node));
+ return PTR_ERR(shmem_node);
+ }
+
+ return dt_device_get_address(shmem_node, 0, addr, size);
+}
+
+/*
+ * Handle Dom0 SCMI specific DT nodes
+ *
+ * Make a decision on copying SCMI specific nodes into Dom0 device tree.
+ * For SCMI multi-agent case:
+ * - shmem nodes will not be copied and generated instead if SCMI
+ * is enabled for Dom0
+ * - scmi node will be copied if SCMI is enabled for Dom0
+ */
+static bool scmi_dt_handle_node(struct domain *d, struct dt_device_node *node)
+{
+ static const struct dt_device_match skip_matches[] __initconst = {
+ DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
+ { /* sentinel */ },
+ };
+ static const struct dt_device_match scmi_matches[] __initconst = {
+ DT_MATCH_PATH("/firmware/scmi"),
+ { /* sentinel */ },
+ };
+
+ if ( !scmi_data.initialized )
+ return false;
+
+ /* always drop shmem */
+ if ( dt_match_node(skip_matches, node) )
+ {
+ dt_dprintk(" Skip scmi shmem\n");
+ return true;
+ }
+
+ /* drop scmi if not enabled */
+ if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
+ {
+ dt_dprintk(" Skip scmi node\n");
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Finalize Dom0 SCMI specific DT nodes
+ *
+ * if SCMI is enabled for Dom0:
+ * - generate shmem node
+ * - map SCMI shmem MMIO into Dom0
+ */
+static int scmi_dt_finalize(struct domain *d, void *fdt)
+{
+ __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
+ struct scmi_channel *channel;
+ int nodeoffset;
+ __be32 *cells;
+ __be32 val;
+ char buf[64];
+ int res, rc;
+
+ if ( !sci_domain_is_enabled(d) )
+ return 0;
+
+ channel = d->arch.sci_data;
+
+ /*
+ * Replace "arm,smc-id" with proper value assigned for Dom0 SCMI channel
+ */
+ nodeoffset = fdt_node_offset_by_compatible(fdt, -1, "arm,scmi-smc");
+ if ( nodeoffset < 0 )
+ return -ENODEV;
+
+ cells = (__be32 *)&val;
+ dt_set_cell(&cells, 1, channel->func_id);
+ res = fdt_setprop_inplace(fdt, nodeoffset, "arm,smc-id", &val, sizeof(val));
+ if ( res )
+ return -EINVAL;
+
+ /*
+ * All SCMI shmem nodes should be removed from Dom0 DT at this point, so
+ * the shmem node for Dom0 need to be generated from SCMI channel assigned
+ * to Dom0.
+ * The original SCMI shmem node from platform DT is used by Xen SCMI driver
+ * itself as privileged channel (agent_id=0) to manage other SCMI
+ * agents (domains).
+ */
+ snprintf(buf, sizeof(buf), "scmi-shmem@%lx", channel->paddr);
+
+ res = fdt_begin_node(fdt, buf);
+ if ( res )
+ return res;
+
+ res = fdt_property_string(fdt, "compatible", "arm,scmi-shmem");
+ if ( res )
+ return res;
+
+ cells = ®[0];
+
+ dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+ channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
+
+ res = fdt_property(fdt, "reg", reg, sizeof(reg));
+ if ( res )
+ return res;
+
+ res = fdt_property_cell(fdt, "phandle", scmi_data.shmem_phandle);
+ if ( res )
+ return res;
+
+ res = fdt_end_node(fdt);
+ if ( res )
+ return res;
+
+ /*
+ * Map SCMI shmem into Dom0 here as shmem nodes are excluded from
+ * generic Dom0 DT processing
+ */
+ res = iomem_permit_access(d, paddr_to_pfn(channel->paddr),
+ paddr_to_pfn(channel->paddr +
+ SCMI_SHMEM_MAPPED_SIZE - 1));
+ if ( res )
+ return res;
+
+ res = map_regions_p2mt(d, gaddr_to_gfn(channel->paddr),
+ PFN_UP(SCMI_SHMEM_MAPPED_SIZE),
+ maddr_to_mfn(channel->paddr), p2m_mmio_direct_nc);
+ if ( res )
+ {
+ rc = iomem_deny_access(d, paddr_to_pfn(channel->paddr),
+ paddr_to_pfn(channel->paddr +
+ SCMI_SHMEM_MAPPED_SIZE - 1));
+ if ( rc )
+ printk(XENLOG_ERR "scmi: Unable to deny iomem access , err = %d\n",
+ rc);
+ }
+
+ return res;
+}
+
+static int scmi_assign_device(uint32_t agent_id, uint32_t device_id,
+ uint32_t flags)
+{
+ struct scmi_msg_base_set_device_permissions_a2p tx;
+ struct scmi_channel *channel;
+ scmi_msg_header_t hdr;
+ int ret;
+
+ channel = get_channel_by_id(HYP_CHANNEL);
+ if ( !channel )
+ return -EINVAL;
+
+ hdr.id = SCMI_BASE_SET_DEVICE_PERMISSIONS;
+ hdr.type = 0;
+ hdr.protocol = SCMI_BASE_PROTOCOL;
+
+ tx.agent_id = agent_id;
+ tx.device_id = device_id;
+ tx.flags = flags;
+
+ ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
+ if ( ret == -EOPNOTSUPP )
+ return 0;
+
+ return ret;
+}
+
+static int scmi_dt_assign_device(struct domain *d,
+ struct dt_phandle_args *ac_spec)
+{
+ struct scmi_channel *agent_channel;
+ uint32_t scmi_device_id = ac_spec->args[0];
+ int ret;
+
+ if ( !d->arch.sci_data )
+ return 0;
+
+ /* The access-controllers is specified for DT dev, but it's not a SCMI */
+ if ( ac_spec->np != scmi_data.dt_dev )
+ return 0;
+
+ agent_channel = d->arch.sci_data;
+
+ spin_lock(&agent_channel->lock);
+
+ ret = scmi_assign_device(agent_channel->agent_id, scmi_device_id,
+ SCMI_BASE_DEVICE_ACCESS_ALLOW);
+ if ( ret )
+ {
+ printk(XENLOG_ERR
+ "scmi: could not assign dev for %pd agent:%d dev_id:%u (%d)",
+ d, agent_channel->agent_id, scmi_device_id, ret);
+ }
+
+ spin_unlock(&agent_channel->lock);
+ return ret;
+}
+
+static __init int collect_agents(struct dt_device_node *scmi_node)
+{
+ const struct dt_device_node *chosen_node;
+ const __be32 *prop;
+ uint32_t len, i;
+
+ chosen_node = dt_find_node_by_path("/chosen");
+ if ( !chosen_node )
+ {
+ printk(XENLOG_ERR "scmi: chosen node not found\n");
+ return -ENOENT;
+ }
+
+ prop = dt_get_property(chosen_node, SCMI_SECONDARY_AGENTS, &len);
+ if ( !prop )
+ {
+ printk(XENLOG_WARNING "scmi: No %s property found\n",
+ SCMI_SECONDARY_AGENTS);
+ return -ENODEV;
+ }
+
+ if ( len % (3 * sizeof(uint32_t)) )
+ {
+ printk(XENLOG_ERR "scmi: Invalid length of %s property: %d\n",
+ SCMI_SECONDARY_AGENTS, len);
+ return -EINVAL;
+ }
+
+ for ( i = 0; i < len / (3 * sizeof(uint32_t)); i++ )
+ {
+ uint32_t agent_id = be32_to_cpu(*prop++);
+ uint32_t smc_id = be32_to_cpu(*prop++);
+ uint32_t shmem_phandle = be32_to_cpu(*prop++);
+ struct dt_device_node *node = dt_find_node_by_phandle(shmem_phandle);
+ u64 addr, size;
+ int ret;
+
+ if ( !node )
+ {
+ printk(XENLOG_ERR "scmi: Could not find shmem node for agent %d\n",
+ agent_id);
+ return -EINVAL;
+ }
+
+ ret = dt_device_get_address(node, 0, &addr, &size);
+ if ( ret )
+ {
+ printk(XENLOG_ERR
+ "scmi: Could not read shmem address for agent %d: %d",
+ agent_id, ret);
+ return ret;
+ }
+
+ if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
+ {
+ printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
+ return -EINVAL;
+ }
+
+ ret = PTR_RET(smc_create_channel(agent_id, smc_id, addr));
+ if ( ret )
+ {
+ printk(XENLOG_ERR "scmi: Could not create channel for agent %d: %d",
+ agent_id, ret);
+ return ret;
+ }
+
+ printk(XENLOG_DEBUG "scmi: Agent %d SMC %X addr %lx\n", agent_id,
+ smc_id, addr);
+ }
+
+ return 0;
+}
+
+static int scmi_domain_init(struct domain *d,
+ struct xen_domctl_createdomain *config)
+{
+ struct scmi_channel *channel;
+ int ret;
+
+ if ( !scmi_data.initialized )
+ return 0;
+
+ /*
+ * Special case for Dom0 - the SCMI support is enabled basing on
+ * "dom0_sci_agent_id" Xen command line parameter
+ */
+ if ( is_hardware_domain(d) )
+ {
+ if ( opt_dom0_scmi_agent_id != SCMI_AGENT_ID_INVALID )
+ {
+ config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
+ config->arch.arm_sci_agent_id = opt_dom0_scmi_agent_id;
+ }
+ else
+ config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
+ }
+
+ if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE )
+ return 0;
+
+ channel = acquire_scmi_channel(d, config->arch.arm_sci_agent_id);
+ if ( IS_ERR(channel) )
+ {
+ printk(XENLOG_ERR
+ "scmi: Failed to acquire SCMI channel for agent_id %u: %ld\n",
+ config->arch.arm_sci_agent_id, PTR_ERR(channel));
+ return PTR_ERR(channel);
+ }
+
+ printk(XENLOG_INFO
+ "scmi: Acquire channel id = 0x%x, domain_id = %d paddr = 0x%lx\n",
+ channel->agent_id, channel->domain_id, channel->paddr);
+
+ /*
+ * Dom0 (if present) needs to have an access to the guest memory range
+ * to satisfy iomem_access_permitted() check in XEN_DOMCTL_iomem_permission
+ * domctl.
+ */
+ if ( hardware_domain && !is_hardware_domain(d) )
+ {
+ ret = iomem_permit_access(hardware_domain, paddr_to_pfn(channel->paddr),
+ paddr_to_pfn(channel->paddr + PAGE_SIZE - 1));
+ if ( ret )
+ goto error;
+ }
+
+ d->arch.sci_data = channel;
+ d->arch.sci_enabled = true;
+
+ return 0;
+
+error:
+ relinquish_scmi_channel(channel);
+ return ret;
+}
+
+int scmi_domain_sanitise_config(struct xen_domctl_createdomain *config)
+{
+ if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
+ config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA )
+ {
+ dprintk(XENLOG_INFO, "scmi: Unsupported ARM_SCI type\n");
+ return -EINVAL;
+ }
+ else if ( config->arch.arm_sci_type ==
+ XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA &&
+ config->arch.arm_sci_agent_id == 0 )
+ {
+ dprintk(XENLOG_INFO,
+ "scmi: A zero ARM SCMI agent_id is not supported\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int scmi_relinquish_resources(struct domain *d)
+{
+ int ret;
+ struct scmi_channel *channel, *agent_channel;
+ scmi_msg_header_t hdr;
+ struct scmi_msg_base_reset_agent_cfg_a2p tx;
+
+ if ( !d->arch.sci_data )
+ return 0;
+
+ agent_channel = d->arch.sci_data;
+
+ spin_lock(&agent_channel->lock);
+ tx.agent_id = agent_channel->agent_id;
+ spin_unlock(&agent_channel->lock);
+
+ channel = get_channel_by_id(HYP_CHANNEL);
+ if ( !channel )
+ {
+ printk(XENLOG_ERR
+ "scmi: Unable to get Hypervisor scmi channel for domain %d\n",
+ d->domain_id);
+ return -EINVAL;
+ }
+
+ hdr.id = SCMI_BASE_RESET_AGENT_CONFIGURATION;
+ hdr.type = 0;
+ hdr.protocol = SCMI_BASE_PROTOCOL;
+
+ tx.flags = 0;
+
+ ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
+ if ( ret == -EOPNOTSUPP )
+ return 0;
+
+ return ret;
+}
+
+static void scmi_domain_destroy(struct domain *d)
+{
+ struct scmi_channel *channel;
+
+ if ( !d->arch.sci_data )
+ return;
+
+ channel = d->arch.sci_data;
+ spin_lock(&channel->lock);
+
+ relinquish_scmi_channel(channel);
+ printk(XENLOG_DEBUG "scmi: Free domain %d\n", d->domain_id);
+
+ d->arch.sci_data = NULL;
+ d->arch.sci_enabled = true;
+
+ spin_unlock(&channel->lock);
+}
+
+static bool scmi_handle_call(struct cpu_user_regs *regs)
+{
+ uint32_t fid = (uint32_t)get_user_reg(regs, 0);
+ struct scmi_channel *agent_channel;
+ struct domain *d = current->domain;
+ struct arm_smccc_res resp;
+ bool res = false;
+
+ if ( !sci_domain_is_enabled(d) )
+ return false;
+
+ agent_channel = d->arch.sci_data;
+ spin_lock(&agent_channel->lock);
+
+ if ( agent_channel->func_id != fid )
+ {
+ res = false;
+ goto unlock;
+ }
+
+ arm_smccc_1_1_smc(fid,
+ get_user_reg(regs, 1),
+ get_user_reg(regs, 2),
+ get_user_reg(regs, 3),
+ get_user_reg(regs, 4),
+ get_user_reg(regs, 5),
+ get_user_reg(regs, 6),
+ get_user_reg(regs, 7),
+ &resp);
+
+ set_user_reg(regs, 0, resp.a0);
+ set_user_reg(regs, 1, resp.a1);
+ set_user_reg(regs, 2, resp.a2);
+ set_user_reg(regs, 3, resp.a3);
+ res = true;
+unlock:
+ spin_unlock(&agent_channel->lock);
+
+ return res;
+}
+
+static const struct sci_mediator_ops scmi_ops = {
+ .domain_init = scmi_domain_init,
+ .domain_destroy = scmi_domain_destroy,
+ .relinquish_resources = scmi_relinquish_resources,
+ .handle_call = scmi_handle_call,
+ .dom0_dt_handle_node = scmi_dt_handle_node,
+ .dom0_dt_finalize = scmi_dt_finalize,
+ .domain_sanitise_config = scmi_domain_sanitise_config,
+ .assign_dt_device = scmi_dt_assign_device,
+};
+
+static int __init scmi_check_smccc_ver(void)
+{
+ if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
+ {
+ printk(XENLOG_WARNING
+ "scmi: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
+ return -ENOSYS;
+ }
+
+ return 0;
+}
+
+static __init int scmi_probe(struct dt_device_node *scmi_node, const void *data)
+{
+ u64 addr, size;
+ int ret, i;
+ struct scmi_channel *channel, *agent_channel;
+ int n_agents;
+ scmi_msg_header_t hdr;
+ struct scmi_msg_base_attributes_p2a rx;
+
+ ASSERT(scmi_node != NULL);
+
+ INIT_LIST_HEAD(&scmi_data.channel_list);
+ spin_lock_init(&scmi_data.channel_list_lock);
+
+ if ( !acpi_disabled )
+ {
+ printk(XENLOG_WARNING "scmi: is not supported when using ACPI\n");
+ return -EINVAL;
+ }
+
+ ret = scmi_check_smccc_ver();
+ if ( ret )
+ return ret;
+
+ if ( !dt_property_read_u32(scmi_node, "arm,smc-id", &scmi_data.func_id) )
+ {
+ printk(XENLOG_ERR "scmi: unable to read smc-id from DT\n");
+ return -ENOENT;
+ }
+
+ /* save shmem phandle and re-use it fro Dom0 DT shmem node */
+ if ( !dt_property_read_u32(scmi_node, "shmem", &scmi_data.shmem_phandle) )
+ {
+ printk(XENLOG_ERR "scmi: unable to read shmem phandle from DT\n");
+ return -ENOENT;
+ }
+
+ ret = scmi_dt_read_hyp_channel_addr(scmi_node, &addr, &size);
+ if ( IS_ERR_VALUE(ret) )
+ return -ENOENT;
+
+ if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
+ {
+ printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
+ return -EINVAL;
+ }
+
+ scmi_data.dt_dev = scmi_node;
+
+ channel = smc_create_channel(HYP_CHANNEL, scmi_data.func_id, addr);
+ if ( IS_ERR(channel) )
+ goto out;
+
+ ret = map_channel_memory(channel);
+ if ( ret )
+ goto out;
+
+ channel->domain_id = DOMID_XEN;
+
+ hdr.id = SCMI_BASE_PROTOCOL_ATTIBUTES;
+ hdr.type = 0;
+ hdr.protocol = SCMI_BASE_PROTOCOL;
+
+ ret = do_smc_xfer(channel, &hdr, NULL, 0, &rx, sizeof(rx));
+ if ( ret )
+ goto error;
+
+ n_agents = SCMI_FIELD_GET(SCMI_BASE_ATTR_NUM_AGENT, rx.attributes);
+ printk(XENLOG_DEBUG "scmi: Got agent count %d\n", n_agents);
+
+ ret = collect_agents(scmi_node);
+ if ( ret )
+ goto error;
+
+ i = 1;
+
+ list_for_each_entry(agent_channel, &scmi_data.channel_list, list)
+ {
+ struct scmi_msg_base_discover_agent_p2a da_rx;
+ struct scmi_msg_base_discover_agent_a2p da_tx;
+
+ ret = map_channel_memory(agent_channel);
+ if ( ret )
+ goto error;
+
+ hdr.id = SCMI_BASE_DISCOVER_AGENT;
+ hdr.type = 0;
+ hdr.protocol = SCMI_BASE_PROTOCOL;
+
+ da_tx.agent_id = agent_channel->agent_id;
+
+ ret = do_smc_xfer(agent_channel, &hdr, &da_tx, sizeof(da_tx), &da_rx,
+ sizeof(da_rx));
+ if ( agent_channel->domain_id != DOMID_XEN )
+ unmap_channel_memory(agent_channel);
+ if ( ret )
+ goto error;
+
+ printk(XENLOG_DEBUG "id=0x%x name=%s\n", da_rx.agent_id, da_rx.name);
+
+ agent_channel->agent_id = da_rx.agent_id;
+
+ if ( i > n_agents )
+ break;
+
+ i++;
+ }
+
+ ret = sci_register(&scmi_ops);
+ if ( ret )
+ {
+ printk(XENLOG_ERR "SCMI: mediator already registered (ret = %d)\n",
+ ret);
+ return ret;
+ }
+
+ scmi_data.initialized = true;
+ goto out;
+
+error:
+ unmap_channel_memory(channel);
+ free_channel_list();
+out:
+ return ret;
+}
+
+static const struct dt_device_match scmi_smc_match[] __initconst = {
+ DT_MATCH_COMPATIBLE("arm,scmi-smc"),
+ { /* sentinel */ },
+};
+
+DT_DEVICE_START(scmi_smc_ma, "SCMI SMC MEDIATOR", DEVICE_FIRMWARE)
+ .dt_match = scmi_smc_match,
+ .init = scmi_probe,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 095b1a23e3..30e46de6d7 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -329,6 +329,7 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
#define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC 1
+#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA 2
struct xen_arch_domainconfig {
/* IN/OUT */
@@ -355,6 +356,8 @@ struct xen_arch_domainconfig {
uint32_t clock_frequency;
/* IN */
uint8_t arm_sci_type;
+ /* IN */
+ uint8_t arm_sci_agent_id;
};
#endif /* __XEN__ || __XEN_TOOLS__ */
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
` (6 preceding siblings ...)
2025-05-19 15:50 ` [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver Oleksii Moisieiev
@ 2025-05-19 15:50 ` Oleksii Moisieiev
2025-05-23 20:19 ` Stefano Stabellini
2025-06-05 22:40 ` [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Julien Grall
8 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-05-19 15:50 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Julien Grall, Michal Orzel, Oleksii Moisieiev,
Roger Pau Monné, Stefano Stabellini, Volodymyr Babchuk,
Grygorii Strashko
From: Grygorii Strashko <grygorii_strashko@epam.com>
Proposal description to add separate SCMI DT node for Xen management agent
under "chosen" or xen-config node, like Hyperlaunch "xen,config".
This proposal introduces a new approach to the Xen multi-domain
configuration, where all Xen-specific configuration has been moved
under the "/chosen" node. This requires less Dom0 device tree
manipulation and isolates Xen configuration from domain configuration.
This approach provides the following device tree (DT) parameters:
- "xen,scmi-secondary-agents": A Xen-specific parameter under the
"/chosen" node, which describes the SCMI agent configuration for
the domains.
- the SCMI configuration for Xen (privileged agent) and the shared
memory configuration for all agents are provided under the "/chosen"
node and are used strictly by Xen for its initial configuration.
- the scmi_shm and SCMI configuration for Dom0 are placed in the
"/firmware/scmi" node so that they can be moved to Dom0 without
any changes.
This configuration allows the use of Xen-specific nodes to provide
information strictly needed by Xen while using the default SCMI
configuration for Dom0 and other domains. As a result, no additional
bindings need to be introduced to the device tree.
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
---
.../arm/firmware/arm-scmi-proposal.rst | 224 ++++++++++++++++++
1 file changed, 224 insertions(+)
create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
new file mode 100644
index 0000000000..fcc2ed2b65
--- /dev/null
+++ b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
@@ -0,0 +1,224 @@
+
+Proposal for SCMI multi-agent driver bindings
+=============================================
+
+Now the Xen configuration for SCMI multi-agent support is done in a bit complicated way, especially
+from SCMI multi-agent driver initialization and Dom0 DT manipulation point of view.
+Also it does not take into account future requirements to support SCP SCMI FW.
+
+To enable SCMI multi-agent user need:
+
+* take host DT with basic SCMI enabled
+* add SCMI shared-memory nodes for all agents
+* update SCMI node to point on SCMI Xen management channel (``[smc-id, shmem]``)
+* add "xen,scmi-secondary-agents" property to the "\chosen" node
+
+.. code::
+
+ chosen {
+ xen,scmi-secondary-agents = <
+ 1 0x82000003 &scmi_shm_1
+ 2 0x82000004 &scmi_shm_2
+ 3 0x82000005 &scmi_shm_3
+ 4 0x82000006 &scmi_shm_4>;
+ }
+
+ /{
+ // SCMI shared-memory nodes for all agents
+ scmi_shm_0 : sram@47ff0000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff0000 0x0 0x1000>;
+ };
+ scmi_shm_1: sram@47ff1000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff1000 0x0 0x1000>;
+ };
+ scmi_shm_2: sram@47ff2000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff2000 0x0 0x1000>;
+ };
+ scmi_shm_3: sram@47ff3000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff3000 0x0 0x1000>;
+ };
+ scmi_shm_4: sram@47ff4000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff4000 0x0 0x1000>;
+ };
+
+ firmware {
+ scmi: scmi {
+ compatible = "arm,scmi-smc";
+ arm, smc - id = <0x82000002>; <--- Xen management agent channel "smc-id"
+ #address-cells = < 1>;
+ #size-cells = < 0>;
+ #access-controller-cells = < 1>;
+ shmem = <&scmi_shm_0>; <--- Xen management agent channel "shmem"
+
+ protocol@X{
+ };
+ };
+ };
+ }
+
+Important thing to note is that all information about multi-channel support is strictly Xen specific.
+
+During initialization the SCMI multi-agent driver uses Host DT SCMI node and
+"xen,scmi-secondary-agents" property to init itself and then, during Dom0 creation, manipulates
+Dom0 DT to remove Xen specific SCMI info and update dom0 SCMI nodes with Dom0 SCMI agent specific
+information.
+
+There are two negative points:
+
+1) Double DT modification - one is user to set up SCMI Xen support in Host DT, second -
+ Dom0 DT manipulation.
+2) In case of future support of mailbox shared-memory transport there could be up to 4 mailboxes and
+ up to 2 shared-memories per SCMI agent channel.
+
+Hence SCMI multi-agent support is Xen specific knowledge there is a proposal to add it as Xen
+specific DT definitions and so minimize Host and Dom0 DT manipulations.
+Those definitions can be added in "/chosen" or, ideally, in "xen,config" node (like in Hyperlaunch design).
+
+The SCMI binding stays generic, just two SCMI nodes defined - one for Xen management channel and
+one for Host Dom0 OSPM.
+
+Example of using "chosen" for configuration:
+
+.. code::
+
+ /{
+
+ chosen {
+ ...
+
+ // Xen SCMI management channel
+ scmi_shm_0 : sram@47ff0000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff0000 0x0 0x1000>;
+ };
+ scmi_xen: scmi {
+ compatible = "arm,scmi-smc";
+ arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
+ #address-cells = < 1>;
+ #size-cells = < 0>;
+ #access-controller-cells = < 1>;
+ shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
+ };
+
+ // SCMI multi-agent configuration
+ scmi_shm_2: sram@47ff2000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff2000 0x0 0x1000>;
+ };
+ scmi_shm_3: sram@47ff3000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff3000 0x0 0x1000>;
+ };
+ scmi_shm_4: sram@47ff4000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff4000 0x0 0x1000>;
+ };
+ xen,scmi-secondary-agents = <
+ 1 0x82000003 &scmi_shm
+ 2 0x82000004 &scmi_shm_2
+ 3 0x82000005 &scmi_shm_3
+ 4 0x82000006 &scmi_shm_4>;
+ };
+
+ // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
+ scmi_shm: sram@47ff1000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff1000 0x0 0x1000>;
+ };
+
+ firmware {
+ scmi: scmi {
+ compatible = "arm,scmi-smc";
+ arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
+ #address-cells = < 1>;
+ #size-cells = < 0>;
+ shmem = <&scmi_shm>; <--- Host OSPM agent shmem
+
+ protocol@X{
+ };
+ };
+ };
+ }
+
+
+In the above case:
+
+1) Xen SCMI multi-agent can be probed with DT configuration from "chosen" (or special "xen,config")
+ node and all Xen related nodes can be easily dropped from Dom0 DT.
+2) Host SCMI OSPM channel DT nodes can be copied to Dom0 DT without changes if SCMI enabled for it.
+3) Future support for mailbox shared-memory transport (SCP SCMI FW) can be simplified as no more
+ manipulation required with Dom0 SCMI "arm,smc-id" and "shmem" DT properties.
+
+
+Example of using "xen,config" for configuration:
+
+.. code::
+
+ hypervisor {
+ compatible = “hypervisor,xen”
+
+ // Configuration container
+ config {
+ compatible = "xen,config";
+ ...
+
+ // Xen SCMI management channel
+ scmi_shm_0 : sram@47ff0000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff0000 0x0 0x1000>;
+ };
+ scmi_xen: scmi {
+ compatible = "arm,scmi-smc";
+ arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
+ #address-cells = < 1>;
+ #size-cells = < 0>;
+ #access-controller-cells = < 1>;
+ shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
+ };
+
+ // SCMI multi-agent configuration
+ scmi_shm_2: sram@47ff2000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff2000 0x0 0x1000>;
+ };
+ scmi_shm_3: sram@47ff3000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff3000 0x0 0x1000>;
+ };
+ scmi_shm_4: sram@47ff4000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff4000 0x0 0x1000>;
+ };
+ xen,scmi-secondary-agents = <
+ 1 0x82000003 &scmi_shm
+ 2 0x82000004 &scmi_shm_2
+ 3 0x82000005 &scmi_shm_3
+ 4 0x82000006 &scmi_shm_4>;
+ };
+ };
+
+ /{
+ // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
+ scmi_shm: sram@47ff1000 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x47ff1000 0x0 0x1000>;
+ };
+
+ firmware {
+ scmi: scmi {
+ compatible = "arm,scmi-smc";
+ arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
+ #address-cells = < 1>;
+ #size-cells = < 0>;
+ shmem = <&scmi_shm>; <--- Host OSPM agent shmem
+
+ protocol@X{
+ };
+ };
+ };
+ }
--
2.34.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-05-19 15:50 ` [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu Oleksii Moisieiev
@ 2025-05-19 18:54 ` Jan Beulich
2025-05-22 0:25 ` Stefano Stabellini
1 sibling, 0 replies; 67+ messages in thread
From: Jan Beulich @ 2025-05-19 18:54 UTC (permalink / raw)
To: Oleksii Moisieiev, Grygorii Strashko
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Juergen Gross,
Julien Grall, Michal Orzel, Roger Pau Monné,
Stefano Stabellini, Volodymyr Babchuk,
xen-devel@lists.xenproject.org
On 19.05.2025 17:50, Oleksii Moisieiev wrote:
> --- a/xen/arch/arm/firmware/sci.c
> +++ b/xen/arch/arm/firmware/sci.c
> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
> return 0;
> }
>
> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> +{
> + struct dt_device_node *dev;
> + int ret = 0;
> +
> + switch ( domctl->cmd )
> + {
> + case XEN_DOMCTL_assign_device:
> + ret = -EOPNOTSUPP;
> + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT )
> + break;
> +
> + if ( !cur_mediator )
> + break;
> +
> + if ( !cur_mediator->assign_dt_device )
> + break;
> +
> + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
> + domctl->u.assign_device.u.dt.size, &dev);
> + if ( ret )
> + return ret;
> +
> + ret = sci_assign_dt_device(d, dev);
> + if ( ret )
> + break;
These two lines are pointless when directly followed by ...
> +
> + break;
... this. Misra calls such "dead code" iirc.
> --- a/xen/arch/arm/include/asm/firmware/sci.h
> +++ b/xen/arch/arm/include/asm/firmware/sci.h
> @@ -146,6 +146,14 @@ int sci_dt_finalize(struct domain *d, void *fdt);
> * control" functionality.
> */
> int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
> +
> +/*
> + * SCI domctl handler
> + *
> + * Only XEN_DOMCTL_assign_device is handled for now.
> + */
> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
> #else
>
> static inline bool sci_domain_is_enabled(struct domain *d)
> @@ -195,6 +203,12 @@ static inline int sci_assign_dt_device(struct domain *d,
> return 0;
> }
>
> +static inline int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> +{
> + return 0;
> +}
> +
> #endif /* CONFIG_ARM_SCI */
>
> #endif /* __ASM_ARM_SCI_H */
This being an Arm-specific header, how does ...
> @@ -851,6 +852,24 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> case XEN_DOMCTL_deassign_device:
> case XEN_DOMCTL_get_device_group:
> ret = iommu_do_domctl(op, d, u_domctl);
> +
> + if ( !ret || ret == -EOPNOTSUPP )
> + {
> + int ret1;
> + /*
> + * Add chained handling of assigned DT devices to support
> + * access-controller functionality through SCI framework, so
> + * DT device assign request can be passed to FW for processing and
> + * enabling VM access to requested device.
> + * The access-controller DT device processing is chained after IOMMU
> + * processing and expected to be executed for any DT device
> + * regardless if DT device is protected by IOMMU or not (or IOMMU
> + * is disabled).
> + */
> + ret1 = sci_do_domctl(op, d, u_domctl);
... this compile on non-Arm? I think I said so before: I don't like this
sitting in common code anyway. Is there really no way to put it in Arm-
specific code?
Jan
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 2/8] xen/arm: scmi-smc: update to be used under sci subsystem
2025-05-19 15:50 ` [RFC PATCH v4 2/8] xen/arm: scmi-smc: update to be used under sci subsystem Oleksii Moisieiev
@ 2025-05-19 23:29 ` Stefano Stabellini
0 siblings, 0 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-05-19 23:29 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Stefano Stabellini,
Volodymyr Babchuk, Grygorii Strashko
On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
>
> The introduced SCI (System Control Interface) subsystem provides unified
> interface to integrate in Xen SCI drivers which adds support for ARM
> firmware (EL3, SCP) based software interfaces (like SCMI) that are used in
> system management. The SCI subsystem allows to add drivers for different FW
> interfaces or have different drivers for the same FW interface (for example,
> SCMI with different transports).
>
> This patch updates SCMI over SMC calls handling layer, introduced by
> commit 3e322bef8bc0 ("xen/arm: firmware: Add SCMI over SMC calls handling
> layer"), to be SCI driver:
> - convert to DT device;
> - convert to SCI Xen interface.
>
> There are no functional changes in general, the driver is just adopted
> to the SCI interface.
>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
>
>
>
> xen/arch/arm/firmware/Kconfig | 13 ++-
> xen/arch/arm/firmware/scmi-smc.c | 93 +++++++++++---------
> xen/arch/arm/include/asm/firmware/scmi-smc.h | 41 ---------
> xen/arch/arm/vsmc.c | 5 +-
> xen/include/public/arch-arm.h | 1 +
> 5 files changed, 64 insertions(+), 89 deletions(-)
> delete mode 100644 xen/arch/arm/include/asm/firmware/scmi-smc.h
>
> diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
> index fc7918c7fc..bbf88fbb9a 100644
> --- a/xen/arch/arm/firmware/Kconfig
> +++ b/xen/arch/arm/firmware/Kconfig
> @@ -8,9 +8,18 @@ config ARM_SCI
>
> menu "Firmware Drivers"
>
> +choice
> + prompt "ARM SCI driver type"
> + default SCMI_SMC
> + help
> + Choose which ARM SCI driver to enable.
> +
> +config ARM_SCI_NONE
> + bool "none"
> +
> config SCMI_SMC
> bool "Forward SCMI over SMC calls from hwdom to EL3 firmware"
> - default y
> + select ARM_SCI
> help
> This option enables basic awareness for SCMI calls using SMC as
> doorbell mechanism and Shared Memory for transport ("arm,scmi-smc"
> @@ -18,4 +27,6 @@ config SCMI_SMC
> firmware node is used to trap and forward corresponding SCMI SMCs
> to firmware running at EL3, for calls coming from the hardware domain.
>
> +endchoice
> +
> endmenu
> diff --git a/xen/arch/arm/firmware/scmi-smc.c b/xen/arch/arm/firmware/scmi-smc.c
> index 33473c04b1..13d1137592 100644
> --- a/xen/arch/arm/firmware/scmi-smc.c
> +++ b/xen/arch/arm/firmware/scmi-smc.c
> @@ -9,6 +9,7 @@
> * Copyright 2024 NXP
> */
>
> +#include <asm/device.h>
> #include <xen/acpi.h>
> #include <xen/device_tree.h>
> #include <xen/errno.h>
> @@ -16,12 +17,11 @@
> #include <xen/sched.h>
> #include <xen/types.h>
>
> +#include <asm/firmware/sci.h>
> #include <asm/smccc.h>
> -#include <asm/firmware/scmi-smc.h>
>
> #define SCMI_SMC_ID_PROP "arm,smc-id"
>
> -static bool __ro_after_init scmi_enabled;
> static uint32_t __ro_after_init scmi_smc_id;
>
> /*
> @@ -41,14 +41,11 @@ static bool scmi_is_valid_smc_id(uint32_t fid)
> *
> * Returns true if SMC was handled (regardless of response), false otherwise.
> */
> -bool scmi_handle_smc(struct cpu_user_regs *regs)
> +static bool scmi_handle_smc(struct cpu_user_regs *regs)
> {
> uint32_t fid = (uint32_t)get_user_reg(regs, 0);
> struct arm_smccc_res res;
>
> - if ( !scmi_enabled )
> - return false;
> -
> if ( !scmi_is_valid_smc_id(fid) )
> return false;
>
> @@ -78,49 +75,45 @@ bool scmi_handle_smc(struct cpu_user_regs *regs)
> return true;
> }
>
> -static int __init scmi_check_smccc_ver(void)
> +static int scmi_smc_domain_init(struct domain *d,
> + struct xen_domctl_createdomain *config)
> {
> - if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
> - {
> - printk(XENLOG_WARNING
> - "SCMI: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
> - return -ENOSYS;
> - }
> + if ( !is_hardware_domain(d) )
> + return 0;
>
> + d->arch.sci_enabled = true;
> + printk(XENLOG_DEBUG "SCMI: %pd init\n", d);
> return 0;
> }
>
> -static int __init scmi_dt_init_smccc(void)
> +static void scmi_smc_domain_destroy(struct domain *d)
> {
> - static const struct dt_device_match scmi_ids[] __initconst =
> - {
> - /* We only support "arm,scmi-smc" binding for now */
> - DT_MATCH_COMPATIBLE("arm,scmi-smc"),
> - { /* sentinel */ },
> - };
> - const struct dt_device_node *scmi_node;
> - int ret;
> + if ( !is_hardware_domain(d) )
> + return;
>
> - /* If no SCMI firmware node found, fail silently as it's not mandatory */
> - scmi_node = dt_find_matching_node(NULL, scmi_ids);
> - if ( !scmi_node )
> - return -EOPNOTSUPP;
> + printk(XENLOG_DEBUG "SCMI: %pd destroy\n", d);
> +}
>
> - ret = dt_property_read_u32(scmi_node, SCMI_SMC_ID_PROP, &scmi_smc_id);
> - if ( !ret )
> +static int __init scmi_check_smccc_ver(void)
> +{
> + if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
> {
> - printk(XENLOG_ERR "SCMI: No valid \"%s\" property in \"%s\" DT node\n",
> - SCMI_SMC_ID_PROP, scmi_node->full_name);
> - return -ENOENT;
> + printk(XENLOG_WARNING
> + "SCMI: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
> + return -ENOSYS;
> }
>
> - scmi_enabled = true;
> -
> return 0;
> }
>
> +static const struct sci_mediator_ops scmi_smc_ops = {
> + .handle_call = scmi_handle_smc,
> + .domain_init = scmi_smc_domain_init,
> + .domain_destroy = scmi_smc_domain_destroy,
> +};
> +
> /* Initialize the SCMI layer based on SMCs and Device-tree */
> -static int __init scmi_init(void)
> +static int __init scmi_dom0_init(struct dt_device_node *dev, const void *data)
> {
> int ret;
>
> @@ -134,22 +127,36 @@ static int __init scmi_init(void)
> if ( ret )
> return ret;
>
> - ret = scmi_dt_init_smccc();
> - if ( ret == -EOPNOTSUPP )
> - return ret;
> + ret = dt_property_read_u32(dev, SCMI_SMC_ID_PROP, &scmi_smc_id);
> + if ( !ret )
> + {
> + printk(XENLOG_ERR "SCMI: No valid \"%s\" property in \"%s\" DT node\n",
> + SCMI_SMC_ID_PROP, dt_node_full_name(dev));
> + return -ENOENT;
> + }
> +
> + ret = sci_register(&scmi_smc_ops);
> if ( ret )
> - goto err;
> + {
> + printk(XENLOG_ERR "SCMI: mediator already registered (ret = %d)\n",
> + ret);
> + return ret;
> + }
>
> printk(XENLOG_INFO "Using SCMI with SMC ID: 0x%x\n", scmi_smc_id);
>
> return 0;
> -
> - err:
> - printk(XENLOG_ERR "SCMI: Initialization failed (ret = %d)\n", ret);
> - return ret;
> }
>
> -__initcall(scmi_init);
> +static const struct dt_device_match scmi_smc_match[] __initconst = {
> + DT_MATCH_COMPATIBLE("arm,scmi-smc"),
> + { /* sentinel */ },
> +};
> +
> +DT_DEVICE_START(scmi_smc, "SCMI SMC DOM0", DEVICE_FIRMWARE)
> + .dt_match = scmi_smc_match,
> + .init = scmi_dom0_init,
> +DT_DEVICE_END
>
> /*
> * Local variables:
> diff --git a/xen/arch/arm/include/asm/firmware/scmi-smc.h b/xen/arch/arm/include/asm/firmware/scmi-smc.h
> deleted file mode 100644
> index 6b1a164a40..0000000000
> --- a/xen/arch/arm/include/asm/firmware/scmi-smc.h
> +++ /dev/null
> @@ -1,41 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0-only */
> -/*
> - * xen/arch/arm/include/asm/firmware/scmi-smc.h
> - *
> - * ARM System Control and Management Interface (SCMI) over SMC
> - * Generic handling layer
> - *
> - * Andrei Cherechesu <andrei.cherechesu@nxp.com>
> - * Copyright 2024 NXP
> - */
> -
> -#ifndef __ASM_SCMI_SMC_H__
> -#define __ASM_SCMI_SMC_H__
> -
> -#include <xen/types.h>
> -
> -struct cpu_user_regs;
> -
> -#ifdef CONFIG_SCMI_SMC
> -
> -bool scmi_handle_smc(struct cpu_user_regs *regs);
> -
> -#else
> -
> -static inline bool scmi_handle_smc(struct cpu_user_regs *regs)
> -{
> - return false;
> -}
> -
> -#endif /* CONFIG_SCMI_SMC */
> -
> -#endif /* __ASM_SCMI_H__ */
> -
> -/*
> - * Local variables:
> - * mode: C
> - * c-file-style: "BSD"
> - * c-basic-offset: 4
> - * indent-tabs-mode: nil
> - * End:
> - */
> diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
> index 51b3c02973..b33c69a1c2 100644
> --- a/xen/arch/arm/vsmc.c
> +++ b/xen/arch/arm/vsmc.c
> @@ -21,7 +21,6 @@
> #include <asm/traps.h>
> #include <asm/vpsci.h>
> #include <asm/platform.h>
> -#include <asm/firmware/scmi-smc.h>
>
> /* Number of functions currently supported by Hypervisor Service. */
> #define XEN_SMCCC_FUNCTION_COUNT 3
> @@ -233,7 +232,7 @@ static bool handle_sip(struct cpu_user_regs *regs)
> if ( platform_smc(regs) )
> return true;
>
> - return scmi_handle_smc(regs);
> + return sci_handle_call(regs);
> }
>
> /*
> @@ -301,8 +300,6 @@ static bool vsmccc_handle_call(struct cpu_user_regs *regs)
> break;
> case ARM_SMCCC_OWNER_SIP:
> handled = handle_sip(regs);
> - if ( !handled )
> - handled = sci_handle_call(regs);
> break;
> case ARM_SMCCC_OWNER_TRUSTED_APP ... ARM_SMCCC_OWNER_TRUSTED_APP_END:
> case ARM_SMCCC_OWNER_TRUSTED_OS ... ARM_SMCCC_OWNER_TRUSTED_OS_END:
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 55eed9992c..095b1a23e3 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -328,6 +328,7 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
> #define XEN_DOMCTL_CONFIG_TEE_FFA 2
>
> #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
> +#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC 1
>
> struct xen_arch_domainconfig {
> /* IN/OUT */
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 1/8] xen/arm: add generic SCI subsystem
2025-05-19 15:50 ` [RFC PATCH v4 1/8] xen/arm: add generic SCI subsystem Oleksii Moisieiev
@ 2025-05-19 23:45 ` Stefano Stabellini
0 siblings, 0 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-05-19 23:45 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Stefano Stabellini,
Volodymyr Babchuk, Grygorii Strashko
On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> This patch adds the basic framework for ARM SCI mediator. SCI is System
> Control Interface, which is designed to redirect requests from the Domains
> to ARM specific Firmware (for example SCMI). This will allow the devices,
> passed-through to the different Domains, to access to the System resources
> (such as clocks/resets etc) by sending requests to the firmware.
>
> ARM SCI subsystem allows to implement different SCI drivers to handle
> specific ARM firmware interfaces (like ARM SCMI) and mediate requests
> between the Domains and the Firmware. Also it allows SCI drivers to perform
> proper action during Domain creation/destruction which is vital for
> handling use cases like Domain reboot.
>
> This patch introduces new DEVICE_FIRMWARE device subclass for probing SCI
> drivers basing on device tree, SCI drivers register itself with
> DT_DEVICE_START/END macro. On init - the SCI drivers should register its
> SCI ops with sci_register(). Only one SCI driver can be supported.
>
> At run-time, the following SCI API calls are introduced:
>
> - sci_domain_sanitise_config() called from arch_sanitise_domain_config()
> - sci_domain_init() called from arch_domain_create()
> - sci_relinquish_resources() called from domain_relinquish_resources()
> - sci_domain_destroy() called from arch_domain_destroy()
> - sci_handle_call() called from vsmccc_handle_call()
> - sci_dt_handle_node()
> sci_dt_finalize() called from handle_node() (Dom0 DT)
>
> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
This patch needs a pretty major rebase. But the looks OK to me.
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
>
> Changes in v4:
> - fix SPDX-License
> - rename DEVICE_ARM_SCI DT device class to FIRMWARE_DEVICE
> - move XEN_DOMCTL_assign_device code in separate patch
> - Add documentation for SCI SCMI drivers
>
> MAINTAINERS | 6 +
> xen/arch/arm/device.c | 5 +
> xen/arch/arm/dom0less-build.c | 7 +
> xen/arch/arm/domain.c | 12 +-
> xen/arch/arm/domain_build.c | 8 +
> xen/arch/arm/firmware/Kconfig | 8 +
> xen/arch/arm/firmware/Makefile | 1 +
> xen/arch/arm/firmware/sci.c | 154 ++++++++++++++++++
> xen/arch/arm/include/asm/domain.h | 5 +
> xen/arch/arm/include/asm/firmware/sci.h | 200 ++++++++++++++++++++++++
> xen/arch/arm/vsmc.c | 3 +
> xen/include/asm-generic/device.h | 1 +
> xen/include/public/arch-arm.h | 4 +
> 13 files changed, 413 insertions(+), 1 deletion(-)
> create mode 100644 xen/arch/arm/firmware/sci.c
> create mode 100644 xen/arch/arm/include/asm/firmware/sci.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index c11b82eca9..f5e3c48b96 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -507,6 +507,12 @@ R: George Dunlap <gwd@xenproject.org>
> S: Supported
> F: xen/common/sched/
>
> +SCI MEDIATORS
> +M: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> +S: Supported
> +F: xen/arch/arm/firmware/sci.c
> +F: xen/arch/arm/include/asm/firmware/sci.h
> +
> SEABIOS UPSTREAM
> M: Wei Liu <wl@xen.org>
> S: Supported
> diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
> index 5610cddcba..bdab96a408 100644
> --- a/xen/arch/arm/device.c
> +++ b/xen/arch/arm/device.c
> @@ -13,6 +13,7 @@
> #include <xen/iocap.h>
> #include <xen/lib.h>
>
> +#include <asm/firmware/sci.h>
> #include <asm/setup.h>
>
> int map_irq_to_domain(struct domain *d, unsigned int irq,
> @@ -303,6 +304,10 @@ int handle_device(struct domain *d, struct dt_device_node *dev, p2m_type_t p2mt,
> return res;
> }
> }
> +
> + res = sci_assign_dt_device(d, dev);
> + if ( res )
> + return res;
> }
>
> res = map_device_irqs_to_domain(d, dev, own_device, irq_ranges);
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 49d1f14d65..a09c4c4bd7 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -14,6 +14,7 @@
> #include <asm/arm64/sve.h>
> #include <asm/dom0less-build.h>
> #include <asm/domain_build.h>
> +#include <asm/firmware/sci.h>
> #include <asm/static-memory.h>
> #include <asm/static-shmem.h>
>
> @@ -321,6 +322,10 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo,
> return -EINVAL;
> }
>
> + res = sci_assign_dt_device(kinfo->d, node);
> + if ( res )
> + return res;
> +
> res = map_device_irqs_to_domain(kinfo->d, node, true, NULL);
> if ( res < 0 )
> return res;
> @@ -970,6 +975,8 @@ void __init create_domUs(void)
> if ( !llc_coloring_enabled && llc_colors_str )
> panic("'llc-colors' found, but LLC coloring is disabled\n");
>
> + d_cfg.arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> +
> /*
> * The variable max_init_domid is initialized with zero, so here it's
> * very important to use the pre-increment operator to call
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 3ba959f866..652aeb7a55 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -25,6 +25,7 @@
> #include <asm/platform.h>
> #include <asm/procinfo.h>
> #include <asm/regs.h>
> +#include <asm/firmware/sci.h>
> #include <asm/tee/tee.h>
> #include <asm/vfp.h>
> #include <asm/vgic.h>
> @@ -694,7 +695,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
> return -EINVAL;
> }
>
> - return 0;
> + return sci_domain_sanitise_config(config);
> }
>
> int arch_domain_create(struct domain *d,
> @@ -786,6 +787,9 @@ int arch_domain_create(struct domain *d,
> d->arch.sve_vl = config->arch.sve_vl;
> #endif
>
> + if ( (rc = sci_domain_init(d, config)) != 0 )
> + goto fail;
> +
> return 0;
>
> fail:
> @@ -846,6 +850,7 @@ void arch_domain_destroy(struct domain *d)
> domain_vgic_free(d);
> domain_vuart_free(d);
> free_xenheap_page(d->shared_info);
> + sci_domain_destroy(d);
> #ifdef CONFIG_ACPI
> free_xenheap_pages(d->arch.efi_acpi_table,
> get_order_from_bytes(d->arch.efi_acpi_len));
> @@ -1039,6 +1044,7 @@ enum {
> PROG_p2m_root,
> PROG_p2m,
> PROG_p2m_pool,
> + PROG_sci,
> PROG_done,
> };
>
> @@ -1098,6 +1104,10 @@ int domain_relinquish_resources(struct domain *d)
> ret = relinquish_p2m_mapping(d);
> if ( ret )
> return ret;
> + PROGRESS(sci):
> + ret = sci_relinquish_resources(d);
> + if ( ret )
> + return ret;
>
> PROGRESS(p2m_root):
> /*
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 7b47abade1..36d28b52a4 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -24,6 +24,7 @@
> #include <asm/setup.h>
> #include <asm/tee/tee.h>
> #include <asm/pci.h>
> +#include <asm/firmware/sci.h>
> #include <asm/platform.h>
> #include <asm/psci.h>
> #include <asm/setup.h>
> @@ -1888,6 +1889,9 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
> return 0;
> }
>
> + if ( sci_dt_handle_node(d, node) )
> + return 0;
> +
> /*
> * The vGIC does not support routing hardware PPIs to guest. So
> * we need to skip any node using PPIs.
> @@ -1988,6 +1992,10 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
> if ( res )
> return res;
>
> + res = sci_dt_finalize(d, kinfo->fdt);
> + if ( res )
> + return res;
> +
> /*
> * Create a second memory node to store the ranges covering
> * reserved-memory regions.
> diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
> index 817da745fd..fc7918c7fc 100644
> --- a/xen/arch/arm/firmware/Kconfig
> +++ b/xen/arch/arm/firmware/Kconfig
> @@ -1,3 +1,11 @@
> +config ARM_SCI
> + bool
> + depends on ARM
> + help
> + This option enables generic Arm SCI (System Control Interface) mediators
> + support. It allows domains to control system resources via one of
> + Arm SCI mediators drivers implemented in XEN, like SCMI.
> +
> menu "Firmware Drivers"
>
> config SCMI_SMC
> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
> index a5e4542666..71bdefc24a 100644
> --- a/xen/arch/arm/firmware/Makefile
> +++ b/xen/arch/arm/firmware/Makefile
> @@ -1 +1,2 @@
> +obj-$(CONFIG_ARM_SCI) += sci.o
> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
> new file mode 100644
> index 0000000000..e1522e10e2
> --- /dev/null
> +++ b/xen/arch/arm/firmware/sci.c
> @@ -0,0 +1,154 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Generic part of the SCI (System Control Interface) subsystem.
> + *
> + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> + * Copyright (c) 2025 EPAM Systems
> + */
> +
> +#include <xen/acpi.h>
> +#include <xen/errno.h>
> +#include <xen/init.h>
> +#include <xen/sched.h>
> +#include <xen/types.h>
> +
> +#include <asm/firmware/sci.h>
> +
> +static const struct sci_mediator_ops __read_mostly *cur_mediator;
> +
> +int sci_register(const struct sci_mediator_ops *ops)
> +{
> + if ( cur_mediator )
> + return -EEXIST;
> +
> + if ( !ops->domain_init || !ops->domain_destroy || !ops->handle_call )
> + return -EINVAL;
> +
> + cur_mediator = ops;
> +
> + return 0;
> +};
> +
> +bool sci_handle_call(struct cpu_user_regs *args)
> +{
> + if ( unlikely(!cur_mediator) )
> + return false;
> +
> + return cur_mediator->handle_call(args);
> +}
> +
> +int sci_domain_init(struct domain *d, struct xen_domctl_createdomain *config)
> +{
> + if ( !cur_mediator )
> + return 0;
> +
> + return cur_mediator->domain_init(d, config);
> +}
> +
> +int sci_domain_sanitise_config(struct xen_domctl_createdomain *config)
> +{
> + if ( !cur_mediator )
> + return 0;
> +
> + if ( !cur_mediator->domain_sanitise_config )
> + return 0;
> +
> + return cur_mediator->domain_sanitise_config(config);
> +}
> +
> +void sci_domain_destroy(struct domain *d)
> +{
> + if ( !cur_mediator )
> + return;
> +
> + cur_mediator->domain_destroy(d);
> +}
> +
> +int sci_relinquish_resources(struct domain *d)
> +{
> + if ( !cur_mediator )
> + return 0;
> +
> + if ( !cur_mediator->relinquish_resources )
> + return 0;
> +
> + return cur_mediator->relinquish_resources(d);
> +}
> +
> +bool sci_dt_handle_node(struct domain *d, struct dt_device_node *node)
> +{
> + if ( !cur_mediator )
> + return 0;
> +
> + if ( !cur_mediator->dom0_dt_handle_node )
> + return 0;
> +
> + return cur_mediator->dom0_dt_handle_node(d, node);
> +}
> +
> +int sci_dt_finalize(struct domain *d, void *fdt)
> +{
> + if ( !cur_mediator )
> + return 0;
> +
> + if ( !cur_mediator->dom0_dt_finalize )
> + return 0;
> +
> + return cur_mediator->dom0_dt_finalize(d, fdt);
> +}
> +
> +int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
> +{
> + struct dt_phandle_args ac_spec;
> + int index = 0;
> + int ret;
> +
> + if ( !cur_mediator )
> + return 0;
> +
> + if ( !cur_mediator->assign_dt_device )
> + return 0;
> +
> + while ( !dt_parse_phandle_with_args(dev, "access-controllers",
> + "#access-controller-cells", index,
> + &ac_spec) )
> + {
> + printk(XENLOG_DEBUG "sci: assign device %s to %pd\n",
> + dt_node_full_name(dev), d);
> +
> + ret = cur_mediator->assign_dt_device(d, &ac_spec);
> + if ( ret )
> + return ret;
> +
> + index++;
> + }
> +
> + return 0;
> +}
> +
> +static int __init sci_init(void)
> +{
> + struct dt_device_node *np;
> + unsigned int num_sci = 0;
> + int rc;
> +
> + dt_for_each_device_node(dt_host, np)
> + {
> + rc = device_init(np, DEVICE_FIRMWARE, NULL);
> + if ( !rc && num_sci )
> + {
> + printk(XENLOG_ERR
> + "SCMI: Only one SCI controller is supported. found second %s\n",
> + np->name);
> + return -EOPNOTSUPP;
> + }
> + else if ( !rc )
> + num_sci++;
> + else if ( rc != -EBADF && rc != -ENODEV )
> + return rc;
> + }
> +
> + return 0;
> +}
> +
> +__initcall(sci_init);
> diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
> index f1d72c6e48..fa0898b7cf 100644
> --- a/xen/arch/arm/include/asm/domain.h
> +++ b/xen/arch/arm/include/asm/domain.h
> @@ -118,6 +118,11 @@ struct arch_domain
> #ifdef CONFIG_TEE
> void *tee;
> #endif
> +#ifdef CONFIG_ARM_SCI
> + bool sci_enabled;
> + /* ARM SCI driver's specific data */
> + void *sci_data;
> +#endif
>
> } __cacheline_aligned;
>
> diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
> new file mode 100644
> index 0000000000..71fb54852e
> --- /dev/null
> +++ b/xen/arch/arm/include/asm/firmware/sci.h
> @@ -0,0 +1,200 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Generic ARM SCI (System Control Interface) subsystem.
> + *
> + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> + * Copyright (c) 2025 EPAM Systems
> + */
> +
> +#ifndef __ASM_ARM_SCI_H
> +#define __ASM_ARM_SCI_H
> +
> +#include <xen/lib.h>
> +#include <xen/types.h>
> +#include <xen/device_tree.h>
> +#include <xen/errno.h>
> +#include <xen/sched.h>
> +
> +#ifdef CONFIG_ARM_SCI
> +
> +struct sci_mediator_ops {
> + /*
> + * Called during domain construction. If it is requested to enable
> + * SCI support, so SCI driver can create own structures for the new domain
> + * and inform firmware about new domain (if required).
> + * Mandatory.
> + */
> + int (*domain_init)(struct domain *d,
> + struct xen_domctl_createdomain *config);
> +
> + /*
> + * Called during domain construction. The SCI driver uses
> + * it to sanitize domain SCI configuration parameters.
> + * Optional.
> + */
> + int (*domain_sanitise_config)(struct xen_domctl_createdomain *config);
> +
> + /*
> + * Called during domain destruction, releases all resources, that
> + * were allocated for domain.
> + * Mandatory.
> + */
> + void (*domain_destroy)(struct domain *d);
> +
> + /*
> + * Called during domain destruction to relinquish resources used
> + * by SCI driver itself and request resources releasing from firmware.
> + * Optional.
> + */
> + int (*relinquish_resources)(struct domain *d);
> +
> + /* SMC/HVC Handle callback */
> + bool (*handle_call)(struct cpu_user_regs *regs);
> +
> + /*
> + * Dom0 DT nodes handling callback so SCI driver can detect DT nodes it
> + * need to handle and decide if those nodes need to be provided to Dom0.
> + * Optional.
> + */
> + bool (*dom0_dt_handle_node)(struct domain *d, struct dt_device_node *node);
> +
> + /*
> + * SCI driver callback called at the end of Dom0 DT generation, so
> + * it can perform steps to modify DT to enable/disable SCI
> + * functionality for Dom0.
> + */
> + int (*dom0_dt_finalize)(struct domain *d, void *fdt);
> +
> + /*
> + * SCI driver callback called when DT device is passed through to guest,
> + * so SCI driver can enable device access to the domain if SCI FW provides
> + * Device specific access control functionality.
> + * Optional.
> + */
> + int (*assign_dt_device)(struct domain *d, struct dt_phandle_args *ac_spec);
> +};
> +
> +
> +static inline bool sci_domain_is_enabled(struct domain *d)
> +{
> + return d->arch.sci_enabled;
> +}
> +
> +/*
> + * Register SCI subsystem ops.
> + *
> + * Register SCI drivers operation and so enable SCI functionality.
> + * Only one SCI driver is supported.
> + */
> +int sci_register(const struct sci_mediator_ops *ops);
> +
> +/*
> + * Initialize SCI functionality for domain if configured.
> + *
> + * Initialization routine to enable SCI functionality for the domain.
> + * The SCI configuration data and decision about enabling SCI functionality
> + * for the domain is SCI driver specific.
> + */
> +int sci_domain_init(struct domain *d, struct xen_domctl_createdomain *config);
> +
> +/*
> + * Sanitise domain configuration parameters.
> + *
> + */
> +int sci_domain_sanitise_config(struct xen_domctl_createdomain *config);
> +
> +/*
> + * Destroy SCI domain instance.
> + */
> +void sci_domain_destroy(struct domain *d);
> +
> +/*
> + * Free resources assigned to the certain domain.
> + */
> +int sci_relinquish_resources(struct domain *d);
> +
> +/*
> + * SMC/HVC Handle callback.
> + *
> + * SCI driver acts as SMC/HVC server for the registered domains and
> + * does redirection of the domain calls to the SCI firmware,
> + * such as ARM TF-A or similar.
> + */
> +bool sci_handle_call(struct cpu_user_regs *regs);
> +
> +/*
> + * Dom0 DT nodes handling function.
> + *
> + * Allows SCI driver to detect DT nodes it need to handle and decide if
> + * those nodes need to be provided to Dom0.
> + */
> +bool sci_dt_handle_node(struct domain *d, struct dt_device_node *node);
> +
> +/*
> + * Dom0 DT generation finalize.
> + *
> + * Called at the end of Dom0 DT generation, so SCI driver can perform steps
> + * to modify DT to enable/disable SCI functionality for Dom0.
> + */
> +int sci_dt_finalize(struct domain *d, void *fdt);
> +
> +/*
> + * Assign DT device to domain.
> + *
> + * Called when DT device is passed through to guest, so SCI driver can enable
> + * device access to the domain if SCI FW provides "Device specific access
> + * control" functionality.
> + */
> +int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
> +#else
> +
> +static inline bool sci_domain_is_enabled(struct domain *d)
> +{
> + return false;
> +}
> +
> +static inline int sci_domain_init(struct domain *d,
> + struct xen_domctl_createdomain *config)
> +{
> + return 0;
> +}
> +
> +static inline int
> +sci_domain_sanitise_config(struct xen_domctl_createdomain *config)
> +{
> + return 0;
> +}
> +
> +static inline void sci_domain_destroy(struct domain *d)
> +{}
> +
> +static inline int sci_relinquish_resources(struct domain *d)
> +{
> + return 0;
> +}
> +
> +static inline bool sci_handle_call(struct cpu_user_regs *args)
> +{
> + return false;
> +}
> +
> +static inline bool sci_dt_handle_node(struct domain *d,
> + struct dt_device_node *node)
> +{
> + return false;
> +}
> +
> +static inline int sci_dt_finalize(struct domain *d, void *fdt)
> +{
> + return false;
> +}
> +
> +static inline int sci_assign_dt_device(struct domain *d,
> + struct dt_device_node *dev)
> +{
> + return 0;
> +}
> +
> +#endif /* CONFIG_ARM_SCI */
> +
> +#endif /* __ASM_ARM_SCI_H */
> diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
> index 62d8117a12..51b3c02973 100644
> --- a/xen/arch/arm/vsmc.c
> +++ b/xen/arch/arm/vsmc.c
> @@ -12,6 +12,7 @@
> #include <public/arch-arm/smccc.h>
> #include <asm/cpuerrata.h>
> #include <asm/cpufeature.h>
> +#include <asm/firmware/sci.h>
> #include <asm/monitor.h>
> #include <asm/regs.h>
> #include <asm/smccc.h>
> @@ -300,6 +301,8 @@ static bool vsmccc_handle_call(struct cpu_user_regs *regs)
> break;
> case ARM_SMCCC_OWNER_SIP:
> handled = handle_sip(regs);
> + if ( !handled )
> + handled = sci_handle_call(regs);
> break;
> case ARM_SMCCC_OWNER_TRUSTED_APP ... ARM_SMCCC_OWNER_TRUSTED_APP_END:
> case ARM_SMCCC_OWNER_TRUSTED_OS ... ARM_SMCCC_OWNER_TRUSTED_OS_END:
> diff --git a/xen/include/asm-generic/device.h b/xen/include/asm-generic/device.h
> index 1acd1ba1d8..e96c5558c2 100644
> --- a/xen/include/asm-generic/device.h
> +++ b/xen/include/asm-generic/device.h
> @@ -18,6 +18,7 @@ enum device_class
> DEVICE_IOMMU,
> DEVICE_INTERRUPT_CONTROLLER,
> DEVICE_PCI_HOSTBRIDGE,
> + DEVICE_FIRMWARE,
> /* Use for error */
> DEVICE_UNKNOWN,
> };
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 24840eeaa6..55eed9992c 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -327,6 +327,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
> #define XEN_DOMCTL_CONFIG_TEE_OPTEE 1
> #define XEN_DOMCTL_CONFIG_TEE_FFA 2
>
> +#define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
> +
> struct xen_arch_domainconfig {
> /* IN/OUT */
> uint8_t gic_version;
> @@ -350,6 +352,8 @@ struct xen_arch_domainconfig {
> *
> */
> uint32_t clock_frequency;
> + /* IN */
> + uint8_t arm_sci_type;
> };
> #endif /* __XEN__ || __XEN_TOOLS__ */
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 3/8] xen/arm: scmi-smc: passthrough SCMI SMC to domain, single agent
2025-05-19 15:50 ` [RFC PATCH v4 3/8] xen/arm: scmi-smc: passthrough SCMI SMC to domain, single agent Oleksii Moisieiev
@ 2025-05-20 0:18 ` Stefano Stabellini
0 siblings, 0 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-05-20 0:18 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Stefano Stabellini,
Volodymyr Babchuk, Grygorii Strashko
On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
>
> The commit 3e322bef8bc0 ("xen/arm: firmware: Add SCMI over SMC calls
> handling layer") introduces simple driver which forwards SCMI over SMC
> calls from hwdom/dom0 to EL3 firmware (TF-A) with a single SCMI OSPM agent
> support. While it is working gracefully for hwdom/dom0 use case it doesn't
> cover "thin Dom0 with guest domain, which serves as Driver domain"
> use-case. In this case HW need to be enable in Driver domain and dom0 is
> performing only control functions.
>
> The EL3 SCMI firmware (TF-A) with a single SCMI OSPM agent support is
> pretty generic case for the default vendors SDK and new platforms.
>
> This patch enables passthrough of SCMI SMC single agent interface to the
> one guest domain serving as Driver domain.
>
> Configure Dom0 to enable SCMI passthrough:
>
> - dom0: add dom0_scmi_smc_passthrough to the Xen Command Line
>
> Enabled SCMI passthrough for guest using toolstack in the following way:
>
> - domD: xl.cfg add "arm_sci" option as below
>
> arm_sci = "type=scmi_smc"
>
> - domD: xl.cfg enable access to the "arm,scmi-shmem"
>
> iomem = [
> "47ff0,1@22001",
> ]
>
> - domD: add SCMI nodes to the Driver domain partial device tree as in the
> below example:
>
> passthrough {
> scmi_shm_0: sram@22001000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x22001000 0x0 0x1000>;
> };
>
> firmware {
> compatible = "simple-bus";
> scmi: scmi {
> compatible = "arm,scmi-smc";
> shmem = <&scmi_shm_0>;
> ...
> }
> }
> }
>
> dom0less case configuration:
>
> - add "xen,sci_type" property for required DomU ("xen,domain") node
>
> xen,sci_type="scmi_smc"
>
> - add scmi nodes to the Driver domain partial device tree the same way
> as above and enable access to the "arm,scmi-shmem" according to
> dom0less documentation. For example:
>
> scmi_shm_0: sram@22001000 {
> compatible = "arm,scmi-shmem";
> reg = <0x00 0x22001000 0x00 0x1000>;
> -> xen,reg = <0x0 0x47ff0000 0x0 0x1000 0x0 0x22001000>;
> -> xen,force-assign-without-iommu;
> };
>
> The SCMI SMC single agent interface can be enabled for one and only one
> domain. In general, the configuration is similar to any other HW
> passthrough, except explicitly enabling SCMI with "arm_sci" xl.cfg option.
>
> Note that "arm,scmi-smc" and "arm,scmi-shmem" nodes will be removed from
> dom0/hwdom DT in case of
>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> ---
>
> Changes in v4:
> - xl.cfg doc
> - fix comments from Stefano Stabellini
> - fix toolstack code as sugested by Anthony PERARD
> - use MATCH_OPTION()
> - move arm_sci struct and cfg params in "arch_arm"
> - add SCMI passthrough for dom0less case
>
> docs/man/xl.cfg.5.pod.in | 34 ++++++++
> docs/misc/arm/device-tree/booting.txt | 15 ++++
> docs/misc/xen-command-line.pandoc | 9 +++
> tools/include/libxl.h | 5 ++
> tools/libs/light/libxl_arm.c | 14 ++++
> tools/libs/light/libxl_types.idl | 10 +++
> tools/xl/xl_parse.c | 36 +++++++++
> xen/arch/arm/dom0less-build.c | 33 +++++++-
> xen/arch/arm/firmware/Kconfig | 4 +-
> xen/arch/arm/firmware/scmi-smc.c | 112 +++++++++++++++++++++++++-
> 10 files changed, 267 insertions(+), 5 deletions(-)
>
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 8e1422104e..1ccf50b8ea 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3092,6 +3092,40 @@ Otherwise, the value specified by the `nr_spis` parameter will be used.
> The number of SPIs should match the highest interrupt ID that will be
> assigned to the domain.
>
> +=item B<arm_sci="ARM_SCI_STRING">
> +
> +Set ARM_SCI specific options for the guest. ARM SCI is System
> +Control Protocol allows domain to manage various functions that are provided
> +by HW platform firmware.
> +
> +B<ARM_SCI_STRING> is a comma separated list of C<KEY=VALUE> settings,
> +from the following list:
> +
> +=over 4
> +
> +=item B<type=STRING>
> +
> +Specifies an ARM SCI type for the guest.
> +
> +=over 4
> +
> +=item B<none>
> +
> +Don't allow guest to use ARM SCI if present on the platform. This is the
> +default value.
> +
> +=item B<scmi_smc>
> +
> +Enables ARM SCMI SMC support for the guest by enabling SCMI over SMC calls
> +forwarding from domain to the EL3 firmware (like Trusted Firmware-A) with a
> +single SCMI OSPM agent support.
> +Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
> +option.
> +
> +=back
> +
> +=back
> +
> =back
>
> =head3 x86
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 9c881baccc..8943c04173 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -281,6 +281,21 @@ with the following properties:
> passed through. This option is the default if this property is missing
> and the user does not provide the device partial device tree for the domain.
>
> +- xen,sci_type
> +
> + A string property specifying an ARM SCI type for the guest.
> +
> + - "none"
> + Don't allow guest to use ARM SCI if present on the platform. This is the
> + default value.
> +
> + - "scmi_smc"
> + Enables ARM SCMI SMC support for the guest by enabling SCMI over SMC calls
> + forwarding from domain to the EL3 firmware (like Trusted Firmware-A) with a
> + single SCMI OSPM agent support.
> + Should be used together with dom0_scmi_smc_passthrough Xen command line
> + option.
> +
> Under the "xen,domain" compatible node, one or more sub-nodes are present
> for the DomU kernel and ramdisk.
>
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 9bbd00baef..8e50f6b7c7 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1082,6 +1082,15 @@ affinities to prefer but be not limited to the specified node(s).
>
> Pin dom0 vcpus to their respective pcpus
>
> +### dom0_scmi_smc_passthrough (ARM)
Please rename this option because written like this is really confusing:
it makes me think we are passing through SCMI to Dom0, while actually it
is the opposite.
I would call this option something like "dom0_hide_scmi_smc" because the
only effect is to remove SCMI_SMC from Dom0. Other names might also be
OK, such as "scmi_smc_passthrough".
> +> `= <boolean>`
> +
> +The option is available when `CONFIG_SCMI_SMC` is compiled in, and allows to
> +enable SCMI SMC single agent interface for any, but only one guest domain,
> +which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
> +SCMI nodes removed from Dom0/hwdom device tree.
> +(for example, thin Dom0 with Driver domain use-case).
> +
> ### dtuart (ARM)
> > `= path [:options]`
>
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index f8fe4afd7d..5fa43637ab 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -313,6 +313,11 @@
> */
> #define LIBXL_HAVE_BUILDINFO_ARCH_NR_SPIS 1
>
> +/*
> + * libxl_domain_build_info has the arch_arm.sci* fields.
> + */
> +#define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SCI 1
> +
> /*
> * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
> * 'soft reset' for domains and there is 'soft_reset' shutdown reason
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 28cea1f643..28ba9eb787 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -222,6 +222,20 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> config->arch.sve_vl = d_config->b_info.arch_arm.sve_vl / 128U;
> }
>
> + switch (d_config->b_info.arch_arm.arm_sci.type) {
> + case LIBXL_ARM_SCI_TYPE_NONE:
> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> + break;
> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> + break;
> + default:
> + LOG(ERROR, "Unknown ARM_SCI type %d",
> + d_config->b_info.arch_arm.arm_sci.type);
> + return ERROR_FAIL;
> + }
> + LOG(DEBUG, " - SCI type=%u", config->arch.arm_sci_type);
> +
> return 0;
> }
>
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index 33c9cfc1a2..aa2190ab5b 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -551,6 +551,15 @@ libxl_sve_type = Enumeration("sve_type", [
> (2048, "2048")
> ], init_val = "LIBXL_SVE_TYPE_DISABLED")
>
> +libxl_arm_sci_type = Enumeration("arm_sci_type", [
> + (0, "none"),
> + (1, "scmi_smc")
> + ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
> +
> +libxl_arm_sci = Struct("arm_sci", [
> + ("type", libxl_arm_sci_type),
> + ])
> +
> libxl_rdm_reserve = Struct("rdm_reserve", [
> ("strategy", libxl_rdm_reserve_strategy),
> ("policy", libxl_rdm_reserve_policy),
> @@ -725,6 +734,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
> ("vuart", libxl_vuart_type),
> ("sve_vl", libxl_sve_type),
> ("nr_spis", uint32),
> + ("arm_sci", libxl_arm_sci),
> ])),
> ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
> ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 9a3679c023..bd22be9d33 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -1284,6 +1284,36 @@ out:
> if (rc) exit(EXIT_FAILURE);
> }
>
> +static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
> + const char *str)
> +{
> + int ret = 0;
> + char *buf2, *ptr;
> + char *oparg;
> +
> + if (NULL == (buf2 = ptr = strdup(str)))
> + return ERROR_NOMEM;
> +
> + ptr = strtok(buf2, ",");
> + while (ptr != NULL)
> + {
> + if (MATCH_OPTION("type", ptr, oparg)) {
> + ret = libxl_arm_sci_type_from_string(oparg, &arm_sci->type);
> + if (ret) {
> + fprintf(stderr, "Unknown ARM_SCI type: %s\n", oparg);
> + ret = ERROR_INVAL;
> + goto parse_error;
> + }
> + }
> +
> + ptr = strtok(NULL, ",");
> + }
> +
> +parse_error:
> + free(buf2);
> + return ret;
> +}
> +
> void parse_config_data(const char *config_source,
> const char *config_data,
> int config_len,
> @@ -2981,6 +3011,12 @@ skip_usbdev:
> if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
> b_info->arch_arm.nr_spis = l;
>
> + if (!xlu_cfg_get_string(config, "arm_sci", &buf, 1)) {
> + if (parse_arm_sci_config(config, &b_info->arch_arm.arm_sci, buf)) {
> + exit(EXIT_FAILURE);
> + }
> + }
> +
> parse_vkb_list(config, d_config);
>
> d_config->virtios = NULL;
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index a09c4c4bd7..0a00f03a25 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -815,6 +815,36 @@ static int __init construct_domU(struct domain *d,
> return rc;
> }
>
> +int __init domu_dt_sci_parse(struct dt_device_node *node,
> + struct xen_domctl_createdomain *d_cfg)
> +{
> + const char *sci_type = NULL;
> + int ret;
> +
> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> +
> + if ( !IS_ENABLED(CONFIG_ARM_SCI) ||
> + !dt_property_read_bool(node, "xen,sci_type") )
> + return 0;
> +
> + ret = dt_property_read_string(node, "xen,sci_type", &sci_type);
> + if ( ret )
> + return ret;
> +
> + if ( !strcmp(sci_type, "none") )
> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> + else if ( !strcmp(sci_type, "scmi_smc") )
> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> + else
> + {
> + printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
> + sci_type, dt_node_name(node));
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> void __init create_domUs(void)
> {
> struct dt_device_node *node;
> @@ -975,7 +1005,8 @@ void __init create_domUs(void)
> if ( !llc_coloring_enabled && llc_colors_str )
> panic("'llc-colors' found, but LLC coloring is disabled\n");
>
> - d_cfg.arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> + if ( domu_dt_sci_parse(node, &d_cfg) )
> + panic("Error getting SCI configuration\n");
>
> /*
> * The variable max_init_domid is initialized with zero, so here it's
> diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
> index bbf88fbb9a..5c5f0880c4 100644
> --- a/xen/arch/arm/firmware/Kconfig
> +++ b/xen/arch/arm/firmware/Kconfig
> @@ -25,7 +25,9 @@ config SCMI_SMC
> doorbell mechanism and Shared Memory for transport ("arm,scmi-smc"
> compatible only). The value of "arm,smc-id" DT property from SCMI
> firmware node is used to trap and forward corresponding SCMI SMCs
> - to firmware running at EL3, for calls coming from the hardware domain.
> + to firmware running at EL3, for calls coming from the hardware domain or
> + driver domain.
> + Use with EL3 firmware which supports only single SCMI OSPM agent.
>
> endchoice
>
> diff --git a/xen/arch/arm/firmware/scmi-smc.c b/xen/arch/arm/firmware/scmi-smc.c
> index 13d1137592..7470a21505 100644
> --- a/xen/arch/arm/firmware/scmi-smc.c
> +++ b/xen/arch/arm/firmware/scmi-smc.c
> @@ -14,6 +14,8 @@
> #include <xen/device_tree.h>
> #include <xen/errno.h>
> #include <xen/init.h>
> +#include <xen/iocap.h>
> +#include <xen/param.h>
> #include <xen/sched.h>
> #include <xen/types.h>
>
> @@ -22,7 +24,11 @@
>
> #define SCMI_SMC_ID_PROP "arm,smc-id"
>
> +static bool __ro_after_init opt_dom0_scmi_smc_passthrough = false;
> +boolean_param("dom0_scmi_smc_passthrough", opt_dom0_scmi_smc_passthrough);
> +
> static uint32_t __ro_after_init scmi_smc_id;
> +static struct domain __read_mostly *scmi_dom;
>
> /*
> * Check if provided SMC Function Identifier matches the one known by the SCMI
> @@ -50,7 +56,7 @@ static bool scmi_handle_smc(struct cpu_user_regs *regs)
> return false;
>
> /* Only the hardware domain should use SCMI calls */
> - if ( !is_hardware_domain(current->domain) )
> + if ( scmi_dom != current->domain )
> {
> gdprintk(XENLOG_WARNING, "SCMI: Unprivileged access attempt\n");
> return false;
> @@ -75,12 +81,45 @@ static bool scmi_handle_smc(struct cpu_user_regs *regs)
> return true;
> }
>
> +static int
> +scmi_smc_domain_sanitise_config(struct xen_domctl_createdomain *config)
> +{
> + if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
> + config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC )
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> static int scmi_smc_domain_init(struct domain *d,
> struct xen_domctl_createdomain *config)
> {
> - if ( !is_hardware_domain(d) )
> + /*
> + * scmi_passthrough is not enabled:
> + * - proceed only for hw_domain
> + * - fail if guest domain has SCMI enabled.
> + */
> + if ( !opt_dom0_scmi_smc_passthrough && !is_hardware_domain(d) )
> + {
> + if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC )
> + return -EINVAL;
> + else
> + return 0;
> + }
> + /*
> + * scmi_passthrough is enabled:
> + * - ignore hw_domain
> + * - proceed only for domain with SCMI enabled.
> + */
> + if ( opt_dom0_scmi_smc_passthrough &&
> + (config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE ||
> + is_hardware_domain(d)) )
> return 0;
>
> + if ( scmi_dom )
> + return -EEXIST;
> +
> + scmi_dom = d;
> d->arch.sci_enabled = true;
> printk(XENLOG_DEBUG "SCMI: %pd init\n", d);
> return 0;
> @@ -88,12 +127,77 @@ static int scmi_smc_domain_init(struct domain *d,
>
> static void scmi_smc_domain_destroy(struct domain *d)
> {
> - if ( !is_hardware_domain(d) )
> + if ( scmi_dom && scmi_dom != d )
> return;
>
> + scmi_dom = NULL;
> + d->arch.sci_enabled = false;
> printk(XENLOG_DEBUG "SCMI: %pd destroy\n", d);
> }
>
> +/*
> + * Handle Dom0 SCMI SMC specific DT nodes
> + *
> + * if dom0_scmi_smc_passthrough=false:
> + * - Copy SCMI nodes into Dom0 device tree.
> + * if dom0_scmi_smc_passthrough=true:
> + * - skip SCMI nodes from Dom0 DT
> + * - give dom0 control access to SCMI shmem MMIO, so SCMI can be passed
> + * through to guest.
Is this so that the xl toolstack can be used? Because I don't think it
would be needed in the dom0less case?
> + */
> +static bool scmi_smc_dt_handle_node(struct domain *d,
> + struct dt_device_node *node)
> +{
> + static const struct dt_device_match shmem_matches[] __initconst = {
> + DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
> + { /* sentinel */ },
> + };
> + static const struct dt_device_match scmi_matches[] __initconst = {
> + DT_MATCH_PATH("/firmware/scmi"),
> + { /* sentinel */ },
> + };
> +
> + /* skip scmi shmem node for dom0 if scmi not enabled */
> + if ( dt_match_node(shmem_matches, node) && !sci_domain_is_enabled(d) )
> + {
> + dt_dprintk("Skip scmi shmem node\n");
> + return true;
> + }
> +
> + /*
> + * skip scmi node for dom0 if scmi not enabled, but give dom0 control
> + * access to SCMI shmem
> + */
> + if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
> + {
> + struct dt_device_node *shmem_node;
> + const __be32 *prop;
> + u64 paddr, size;
> + int ret;
> +
> + /* give dom0 control access to SCMI shmem */
> + prop = dt_get_property(node, "shmem", NULL);
> + if ( !prop )
> + return true;
> +
> + shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
> + if ( !shmem_node )
> + return true;
> +
> + ret = dt_device_get_address(shmem_node, 0, &paddr, &size);
> + if ( ret )
> + return true;
> +
> + ret = iomem_permit_access(d, paddr_to_pfn(paddr),
> + paddr_to_pfn(paddr + size - 1));
> +
> + dt_dprintk("Skip scmi node\n");
> + return true;
> + }
> +
> + return false;
> +}
> +
> static int __init scmi_check_smccc_ver(void)
> {
> if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
> @@ -108,8 +212,10 @@ static int __init scmi_check_smccc_ver(void)
>
> static const struct sci_mediator_ops scmi_smc_ops = {
> .handle_call = scmi_handle_smc,
> + .domain_sanitise_config = scmi_smc_domain_sanitise_config,
> .domain_init = scmi_smc_domain_init,
> .domain_destroy = scmi_smc_domain_destroy,
> + .dom0_dt_handle_node = scmi_smc_dt_handle_node,
> };
>
> /* Initialize the SCMI layer based on SMCs and Device-tree */
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-05-19 15:50 ` [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu Oleksii Moisieiev
2025-05-19 18:54 ` Jan Beulich
@ 2025-05-22 0:25 ` Stefano Stabellini
2025-05-22 6:18 ` Jan Beulich
2025-06-12 11:42 ` Oleksii Moisieiev
1 sibling, 2 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-05-22 0:25 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Stefano Stabellini,
Volodymyr Babchuk, Grygorii Strashko
On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
>
> Add chained handling of assigned DT devices to support access-controller
> functionality through SCI framework, so DT device assign request can be
> passed to FW for processing and enabling VM access to requested device
> (for example, device power management through FW interface like SCMI).
>
> The SCI access-controller DT device processing is chained after IOMMU
> processing and expected to be executed for any DT device regardless of its
> protection by IOMMU (or if IOMMU is disabled).
>
> This allows to pass not only IOMMU protected DT device through
> xl.cfg:"dtdev" property for processing:
>
> dtdev = [
> "/soc/video@e6ef0000", <- IOMMU protected device
> "/soc/i2c@e6508000", <- not IOMMU protected device
> ]
>
> The change is done in two parts:
> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
> not fail if DT device is not protected by IOMMU
> 2) add chained call to sci_do_domctl() in do_domctl()
>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> ---
>
>
>
> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
> xen/common/domctl.c | 19 +++++++++++++
> xen/drivers/passthrough/device_tree.c | 6 ++++
> 4 files changed, 76 insertions(+)
>
> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
> index e1522e10e2..8efd541c4f 100644
> --- a/xen/arch/arm/firmware/sci.c
> +++ b/xen/arch/arm/firmware/sci.c
> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
> return 0;
> }
>
> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> +{
> + struct dt_device_node *dev;
> + int ret = 0;
> +
> + switch ( domctl->cmd )
> + {
> + case XEN_DOMCTL_assign_device:
> + ret = -EOPNOTSUPP;
Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
> + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT )
> + break;
this one
> + if ( !cur_mediator )
> + break;
this one
> + if ( !cur_mediator->assign_dt_device )
> + break;
and also this one? It seems more like an -EINVAL as the caller used a
wrong parameter?
> + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
> + domctl->u.assign_device.u.dt.size, &dev);
> + if ( ret )
> + return ret;
> +
> + ret = sci_assign_dt_device(d, dev);
> + if ( ret )
> + break;
> +
> + break;
> + default:
> + /* do not fail here as call is chained with iommu handling */
It looks like this should be an error
> + break;
> + }
> +
> + return ret;
> +}
> +
> static int __init sci_init(void)
> {
> struct dt_device_node *np;
> diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
> index 71fb54852e..b8d1bc8a62 100644
> --- a/xen/arch/arm/include/asm/firmware/sci.h
> +++ b/xen/arch/arm/include/asm/firmware/sci.h
> @@ -146,6 +146,14 @@ int sci_dt_finalize(struct domain *d, void *fdt);
> * control" functionality.
> */
> int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
> +
> +/*
> + * SCI domctl handler
> + *
> + * Only XEN_DOMCTL_assign_device is handled for now.
> + */
> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
> #else
>
> static inline bool sci_domain_is_enabled(struct domain *d)
> @@ -195,6 +203,12 @@ static inline int sci_assign_dt_device(struct domain *d,
> return 0;
> }
>
> +static inline int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> +{
> + return 0;
> +}
> +
> #endif /* CONFIG_ARM_SCI */
>
> #endif /* __ASM_ARM_SCI_H */
> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
> index 05abb581a0..a74ee92067 100644
> --- a/xen/common/domctl.c
> +++ b/xen/common/domctl.c
> @@ -27,6 +27,7 @@
> #include <xen/vm_event.h>
> #include <xen/monitor.h>
> #include <asm/current.h>
> +#include <asm/firmware/sci.h>
> #include <asm/irq.h>
> #include <asm/page.h>
> #include <asm/p2m.h>
> @@ -851,6 +852,24 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> case XEN_DOMCTL_deassign_device:
> case XEN_DOMCTL_get_device_group:
> ret = iommu_do_domctl(op, d, u_domctl);
> +
> + if ( !ret || ret == -EOPNOTSUPP )
It is better to invert the check:
if ( ret < 0 && ret != -EOPNOTSUPP )
return ret;
> + {
> + int ret1;
> + /*
> + * Add chained handling of assigned DT devices to support
> + * access-controller functionality through SCI framework, so
> + * DT device assign request can be passed to FW for processing and
> + * enabling VM access to requested device.
> + * The access-controller DT device processing is chained after IOMMU
> + * processing and expected to be executed for any DT device
> + * regardless if DT device is protected by IOMMU or not (or IOMMU
> + * is disabled).
> + */
> + ret1 = sci_do_domctl(op, d, u_domctl);
> + if ( ret1 != -EOPNOTSUPP )
> + ret = ret1;
> + }
> break;
>
> case XEN_DOMCTL_get_paging_mempool_size:
> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
> index 075fb25a37..2624767e51 100644
> --- a/xen/drivers/passthrough/device_tree.c
> +++ b/xen/drivers/passthrough/device_tree.c
> @@ -318,6 +318,12 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
> break;
> }
>
> + if ( !dt_device_is_protected(dev) )
> + {
> + ret = 0;
> + break;
> + }
I am concerned about this: previously we would call
iommu_assign_dt_device and the same check at the beginning of
iommu_assign_dt_device would return -EINVAL. Now it is a success.
I am not sure this is appropriate. I wonder if instead we should:
- remove this chunk from the patch
- change the return error for !dt_device_is_protected at the top of
iommu_assign_dt_device from -EINVAL to -EOPNOTSUPP
- this would fall into the same ret != -EOPNOTSUPP check after
iommu_do_domctl
> ret = iommu_assign_dt_device(d, dev);
>
> if ( ret )
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-05-22 0:25 ` Stefano Stabellini
@ 2025-05-22 6:18 ` Jan Beulich
2025-06-12 11:42 ` Oleksii Moisieiev
1 sibling, 0 replies; 67+ messages in thread
From: Jan Beulich @ 2025-05-22 6:18 UTC (permalink / raw)
To: Stefano Stabellini, Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On 22.05.2025 02:25, Stefano Stabellini wrote:
> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>> + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
>> + domctl->u.assign_device.u.dt.size, &dev);
>> + if ( ret )
>> + return ret;
>> +
>> + ret = sci_assign_dt_device(d, dev);
>> + if ( ret )
>> + break;
>> +
>> + break;
>> + default:
>> + /* do not fail here as call is chained with iommu handling */
>
> It looks like this should be an error
The way the hooking is done at the call site, I think it can't be an error
here. But I previously questioned the way that hooking is done.
Jan
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-05-19 15:50 ` [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver Oleksii Moisieiev
@ 2025-05-23 20:06 ` Stefano Stabellini
2025-06-02 7:17 ` Bertrand Marquis
` (2 more replies)
2025-06-18 7:28 ` Julien Grall
1 sibling, 3 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-05-23 20:06 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Stefano Stabellini,
Volodymyr Babchuk, Grygorii Strashko
One question for Bertrand below
On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
> (TF-A) which provides SCMI interface with multi-agnet support, as shown
> below.
>
> +-----------------------------------------+
> | |
> | EL3 TF-A SCMI |
> +-------+--+-------+--+-------+--+-------++
> |shmem0 | |shmem1 | |shmem2 | |shmemX |
> +-----+-+ +---+---+ +--+----+ +---+---+
> smc-id0 | | | |
> agent0 | | | |
> +-----v--------+---------+-----------+----+
> | | | | |
> | | | | |
> +--------------+---------+-----------+----+
> smc-id1 | smc-id2| smc-idX|
> agent1 | agent2 | agentX |
> | | |
> +----v---+ +--v-----+ +--v-----+
> | | | | | |
> | Dom0 | | Dom1 | | DomX |
> | | | | | |
> | | | | | |
> +--------+ +--------+ +--------+
>
> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
> memory transport for every Agent in the system.
>
> The SCMI Agent transport channel defined by pair:
> - smc-id: SMC/HVC id used for Doorbell
> - shmem: shared memory for messages transfer, Xen page aligned,
> p2m_mmio_direct_nc.
>
> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
> multi-agent functionality under Xen:
> - Xen manegement agent: trusted agents that accesses to the Base Protocol
> commands to configure agent specific permissions
> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
> allowed direct HW access. At least one OSPM VM agent has to be provided
> by FW if HW is handled only by Dom0 or Driver Domain.
>
> The EL3 SCMI FW expected to implement following Base protocol messages:
> - BASE_DISCOVER_AGENT
> - BASE_RESET_AGENT_CONFIGURATION (optional)
> - BASE_SET_DEVICE_PERMISSIONS (optional)
>
> The SCI SCMI SMC multi-agent driver implements following functionality:
> - It's initialized based on the Host DT SCMI node (only one SCMI interface
> is supported) which describes Xen management agent SCMI interface.
>
> scmi_shm_0 : sram@47ff0000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff0000 0x0 0x1000>;
> };
> firmware {
> scmi: scmi {
> compatible = "arm,scmi-smc";
> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
some extra spaces, it might be a copy/paste error
> \#address-cells = < 1>;
> \#size-cells = < 0>;
> \#access-controller - cells = < 1>;
> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
>
> protocol@X{
> };
> };
> };
>
> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
>
> chosen {
> xen,scmi-secondary-agents = <
> 1 0x82000003 &scmi_shm_1
> 2 0x82000004 &scmi_shm_2
> 3 0x82000005 &scmi_shm_3
> 4 0x82000006 &scmi_shm_4>;
> }
>
> /{
> scmi_shm_1: sram@47ff1000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff1000 0x0 0x1000>;
> };
> scmi_shm_2: sram@47ff2000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff2000 0x0 0x1000>;
> };
> scmi_shm_3: sram@47ff3000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff3000 0x0 0x1000>;
> };
> }
> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
> this agent_id.
>
> Note that Xen is the only one entry in the system which need to know
> about SCMI multi-agent support.
>
> - It implements the SCI subsystem interface required for configuring and
> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
> SCMI functionality for domain it has to be configured with unique supported
> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
> [smc-id, shmem] defined for this SCMI Agent_id.
> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
> -- zero-copy, the guest domain puts SCMI message in shmem;
> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
> -- the Xen driver catches exception, do checks and synchronously forwards
> it to EL3 FW.
> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
> management agent channel on domain destroy event. This allows to reset
> resources used by domain and so implement use-case like domain reboot.
>
> Dom0 Enable SCMI SMC:
> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
> node according to assigned agent_id.
>
> Guest domains enable SCMI SMC:
> - xl.cfg: add configuration option as below
>
> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
>
> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
> the domain, for example:
>
> iomem = [
> "47ff2,1@22001",
> ]
Looking at the code and the configuration options, it looks like it is
possible to map a scmi-shmem channel at a different address for the
guest. It seems like it would work. Is that correct?
> - DT: add SCMI nodes to the Driver domain partial device tree as in the
> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
>
> passthrough {
> scmi_shm_0: sram@22001000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x22001000 0x0 0x1000>;
> };
>
> firmware {
> compatible = "simple-bus";
> scmi: scmi {
> compatible = "arm,scmi-smc";
> arm,smc-id = <0x82000004>;
> shmem = <&scmi_shm_0>;
> ...
> }
> }
> }
>
> SCMI "4.2.1.1 Device specific access control"
>
> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
> to the Xen SCMI.
>
> &i2c1 {
> access-controllers = <&scmi 0>;
> };
>
> The Dom0 and dom0less domains DT devices will be processed automatically through
> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
> shell be used:
>
> dtdev = [
> "/soc/i2c@e6508000",
> ]
>
> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
>
> [1] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
> [2] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Thanks for the long explanation, great work! I am really looking forward
to have this feature in the tree soon.
> ---
>
> Changes in v4:
> - toolstack comments from Anthony PERARD
> - added dom0less support
> - added doc for "xen,scmi-secondary-agents"
>
> docs/man/xl.cfg.5.pod.in | 13 +
> docs/misc/arm/device-tree/booting.txt | 60 ++
> docs/misc/xen-command-line.pandoc | 9 +
> tools/libs/light/libxl_arm.c | 4 +
> tools/libs/light/libxl_types.idl | 4 +-
> tools/xl/xl_parse.c | 12 +
> xen/arch/arm/dom0less-build.c | 11 +
> xen/arch/arm/domain_build.c | 3 +-
> xen/arch/arm/firmware/Kconfig | 11 +
> xen/arch/arm/firmware/Makefile | 1 +
> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
> xen/arch/arm/firmware/scmi-shmem.h | 45 +
> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
> xen/include/public/arch-arm.h | 3 +
> 15 files changed, 1371 insertions(+), 2 deletions(-)
> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
>
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 1ccf50b8ea..302c46d8bc 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
> option.
>
> +=item B<scmi_smc_multiagent>
> +
> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
> +specified for the guest.
> +
> =back
>
> +=item B<agent_id=NUMBER>
> +
> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
> +existing on a single host must be unique and in the range [1..255].
> +
> =back
>
> =back
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 8943c04173..c8923ab8b2 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -296,6 +296,20 @@ with the following properties:
> Should be used together with dom0_scmi_smc_passthrough Xen command line
> option.
>
> + - "scmi_smc_multiagent"
> +
> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> + SMC calls forwarding from domain to the EL3 firmware (like ARM
> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
> + property.
> +
> +- "xen,sci_agent_id"
> +
> + Specifies a non-zero ARM SCI agent id for the guest. This option is
> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
> + the guest. The agent ids of guest must be unique and in the range [1..255].
> +
> Under the "xen,domain" compatible node, one or more sub-nodes are present
> for the DomU kernel and ramdisk.
>
> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
> guest physical address space. DomU1 is explicitly defined as the owner domain,
> and DomU2 is the borrower domain.
> +
> +SCMI SMC multi-agent support
> +============================
> +
> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
> +according to the SCMI compliant EL3 Firmware specification with
> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
> +the top-level "chosen" node:
> +
> +- xen,scmi-secondary-agents
> +
> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
> + available for Xen. Each Agent defined as triple consisting of:
> + SCMI agent_id,
> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
> +
> +As an example:
> +
> +chosen {
> + xen,scmi-secondary-agents = <
> + 1 0x82000003 &scmi_shm_1
> + 2 0x82000004 &scmi_shm_2
> + 3 0x82000005 &scmi_shm_3
> + 4 0x82000006 &scmi_shm_4>;
> +}
NIT: it should be };
Looking at scmi_probe, collect_agents, and the following SCMI
SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
information?
It looks like we can discover the agend_ids for every channel, I guess
what we need to know is the shmem location for every channel? But the
full list of shmem channel is available below thanks to the scmi-shmem
nodes.
So, we have the list of scmi-shmem anyway, and we can probe the
agent_id. The only parameter left is the smc_id/func_id.
Or maybe smc_id/func_id can be calculated from agent_id?
I am asking mostly because if a user is supposed to add this
xen,scmi-secondary-agents property, where are they supposed to find the
smc_id/func_id information?
It is important that we write down in this document how the user is
expected to find out what 1 is 0x82000003 which is scmi_shm_1.
> +/{
> + scmi_shm_1: sram@47ff1000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> + };
> + scmi_shm_2: sram@47ff2000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> + };
> + scmi_shm_3: sram@47ff3000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> + };
> + scmi_shm_3: sram@47ff4000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> + };
Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
Or are under /firmware? Or are they under /chosen?
I take they are under the top level node together with scmi_shm_0?
Can you please also clarify in the document as well?
> +}
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 8e50f6b7c7..bc3c64d6ec 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
> SCMI nodes removed from Dom0/hwdom device tree.
> (for example, thin Dom0 with Driver domain use-case).
>
> +### dom0_scmi_agent_id (ARM)
> +> `= <integer>`
> +
> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
> +The SCMI will be disabled for Dom0 if this option is not specified
> +(for example, thin Dom0 or dom0less use-cases).
> +The agent ids of domains existing on a single host must be unique.
> +
> ### dtuart (ARM)
> > `= path [:options]`
>
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 28ba9eb787..7712f53cd4 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> break;
> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
> + break;
> default:
> LOG(ERROR, "Unknown ARM_SCI type %d",
> d_config->b_info.arch_arm.arm_sci.type);
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index aa2190ab5b..11e31ce786 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
>
> libxl_arm_sci_type = Enumeration("arm_sci_type", [
> (0, "none"),
> - (1, "scmi_smc")
> + (1, "scmi_smc"),
> + (2, "scmi_smc_multiagent")
> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
>
> libxl_arm_sci = Struct("arm_sci", [
> ("type", libxl_arm_sci_type),
> + ("agent_id", uint8)
> ])
>
> libxl_rdm_reserve = Struct("rdm_reserve", [
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index bd22be9d33..81aa3797e3 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
> }
> }
>
> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
> + unsigned long val = parse_ulong(oparg);
> +
> + if (!val || val > 255) {
> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
> + val);
> + ret = ERROR_INVAL;
> + goto parse_error;
> + }
> + arm_sci->agent_id = val;
> + }
> +
> ptr = strtok(NULL, ",");
> }
>
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 0a00f03a25..43d21eb889 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> else if ( !strcmp(sci_type, "scmi_smc") )
> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
> + {
> + uint32_t agent_id = 0;
> +
> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
> + !agent_id )
shouldn't we check that agent_id <= 255 ?
> + return -EINVAL;
> +
> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> + d_cfg->arch.arm_sci_agent_id = agent_id;
> + }
> else
> {
> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 36d28b52a4..0c9274a2b3 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
> continue;
>
> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") )
> diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
> index 5c5f0880c4..6b051c8ada 100644
> --- a/xen/arch/arm/firmware/Kconfig
> +++ b/xen/arch/arm/firmware/Kconfig
> @@ -29,6 +29,17 @@ config SCMI_SMC
> driver domain.
> Use with EL3 firmware which supports only single SCMI OSPM agent.
>
> +config SCMI_SMC_MA
> + bool "Enable ARM SCMI SMC multi-agent driver"
> + select ARM_SCI
> + help
> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
> + to EL3 firmware (TF-A) which supports multi-agent feature.
> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
> + allowed platform resources through dedicated SMC/HVC Shared memory based
> + transport.
> +
> endchoice
>
> endmenu
> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
> index 71bdefc24a..37927e690e 100644
> --- a/xen/arch/arm/firmware/Makefile
> +++ b/xen/arch/arm/firmware/Makefile
> @@ -1,2 +1,3 @@
> obj-$(CONFIG_ARM_SCI) += sci.o
> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
> new file mode 100644
> index 0000000000..3f4b9c5d6b
> --- /dev/null
> +++ b/xen/arch/arm/firmware/scmi-proto.h
> @@ -0,0 +1,164 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Arm System Control and Management Interface definitions
> + * Version 3.0 (DEN0056C)
> + *
> + * Copyright (c) 2024 EPAM Systems
> + */
> +
> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
NIT: ARM_FIRMWARE_SCMI_PROTO_H
> +#include <xen/stdint.h>
> +
> +#define SCMI_SHORT_NAME_MAX_SIZE 16
> +
> +/* SCMI status codes. See section 4.1.4 */
> +#define SCMI_SUCCESS 0
> +#define SCMI_NOT_SUPPORTED (-1)
> +#define SCMI_INVALID_PARAMETERS (-2)
> +#define SCMI_DENIED (-3)
> +#define SCMI_NOT_FOUND (-4)
> +#define SCMI_OUT_OF_RANGE (-5)
> +#define SCMI_BUSY (-6)
> +#define SCMI_COMMS_ERROR (-7)
> +#define SCMI_GENERIC_ERROR (-8)
> +#define SCMI_HARDWARE_ERROR (-9)
> +#define SCMI_PROTOCOL_ERROR (-10)
> +
> +/* Protocol IDs */
> +#define SCMI_BASE_PROTOCOL 0x10
> +
> +/* Base protocol message IDs */
> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
> +#define SCMI_BASE_DISCOVER_AGENT 0x7
> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
> +
> +typedef struct scmi_msg_header {
> + uint8_t id;
> + uint8_t type;
> + uint8_t protocol;
> + uint32_t status;
> +} scmi_msg_header_t;
> +
> +/* Table 2 Message header format */
> +#define SCMI_HDR_ID GENMASK(7, 0)
> +#define SCMI_HDR_TYPE GENMASK(9, 8)
> +#define SCMI_HDR_PROTO GENMASK(17, 10)
> +
> +#define SCMI_FIELD_GET(_mask, _reg) \
> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
> +#define SCMI_FIELD_PREP(_mask, _val) \
> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
> +
> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
> +{
> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
> +}
> +
> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
> +{
> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
> +}
> +
> +static inline int scmi_to_xen_errno(int scmi_status)
> +{
> + if ( scmi_status == SCMI_SUCCESS )
> + return 0;
> +
> + switch ( scmi_status )
> + {
> + case SCMI_NOT_SUPPORTED:
> + return -EOPNOTSUPP;
> + case SCMI_INVALID_PARAMETERS:
> + return -EINVAL;
> + case SCMI_DENIED:
> + return -EACCES;
> + case SCMI_NOT_FOUND:
> + return -ENOENT;
> + case SCMI_OUT_OF_RANGE:
> + return -ERANGE;
> + case SCMI_BUSY:
> + return -EBUSY;
> + case SCMI_COMMS_ERROR:
> + return -ENOTCONN;
> + case SCMI_GENERIC_ERROR:
> + return -EIO;
> + case SCMI_HARDWARE_ERROR:
> + return -ENXIO;
> + case SCMI_PROTOCOL_ERROR:
> + return -EBADMSG;
> + default:
> + return -EINVAL;
> + }
> +}
> +
> +/* PROTOCOL_VERSION */
> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
> +
> +struct scmi_msg_prot_version_p2a {
> + uint32_t version;
> +} __packed;
> +
> +/* BASE PROTOCOL_ATTRIBUTES */
> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
> +
> +struct scmi_msg_base_attributes_p2a {
> + uint32_t attributes;
> +} __packed;
> +
> +/*
> + * BASE_DISCOVER_AGENT
> + */
> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
> +
> +struct scmi_msg_base_discover_agent_a2p {
> + uint32_t agent_id;
> +} __packed;
> +
> +struct scmi_msg_base_discover_agent_p2a {
> + uint32_t agent_id;
> + char name[SCMI_SHORT_NAME_MAX_SIZE];
> +} __packed;
> +
> +/*
> + * BASE_SET_DEVICE_PERMISSIONS
> + */
> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
> +
> +struct scmi_msg_base_set_device_permissions_a2p {
> + uint32_t agent_id;
> + uint32_t device_id;
> + uint32_t flags;
> +} __packed;
> +
> +/*
> + * BASE_RESET_AGENT_CONFIGURATION
> + */
> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
> +
> +struct scmi_msg_base_reset_agent_cfg_a2p {
> + uint32_t agent_id;
> + uint32_t flags;
> +} __packed;
> +
> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
> new file mode 100644
> index 0000000000..dd613ee0b5
> --- /dev/null
> +++ b/xen/arch/arm/firmware/scmi-shmem.c
> @@ -0,0 +1,173 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
> + *
> + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> + * Copyright (c) 2025 EPAM Systems
> + */
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +#include <asm/io.h>
> +#include <xen/err.h>
> +
> +#include "scmi-proto.h"
> +#include "scmi-shmem.h"
This code is written more generically than the description implies. If
we only want to make SMC calls to TF-A on EL3 and exchange data with it
over shared memory, then I think:
- we don't need the __iomem tag, as there is no MMIO
- we only need a DMB, not a DSB (readl and writel imply DSB, use only
readl_relaxed and writel_relaxed)
On the other hand, if we also want to handle the case where the SCMI
server could be on a separate co-processor, then what this code is doing
is not sufficient because we also need a dcache flush, in addition to
the DSB.
Bertrand, can you double-check?
> +/*
> + * Copy data from IO memory space to "real" memory space.
> + */
> +static void __memcpy_fromio(void *to, const volatile void __iomem *from,
> + size_t count)
> +{
> + while ( count && !IS_ALIGNED((unsigned long)from, 4) )
> + {
> + *(u8 *)to = readb_relaxed(from);
> + from++;
> + to++;
> + count--;
> + }
> +
> + while ( count >= 4 )
> + {
> + *(u32 *)to = readl_relaxed(from);
> + from += 4;
> + to += 4;
> + count -= 4;
> + }
> +
> + while ( count )
> + {
> + *(u8 *)to = readb_relaxed(from);
> + from++;
> + to++;
> + count--;
> + }
> +}
> +
> +/*
> + * Copy data from "real" memory space to IO memory space.
> + */
> +static void __memcpy_toio(volatile void __iomem *to, const void *from,
> + size_t count)
> +{
> + while ( count && !IS_ALIGNED((unsigned long)to, 4) )
> + {
> + writeb_relaxed(*(u8 *)from, to);
> + from++;
> + to++;
> + count--;
> + }
> +
> + while ( count >= 4 )
> + {
> + writel_relaxed(*(u32 *)from, to);
> + from += 4;
> + to += 4;
> + count -= 4;
> + }
> +
> + while ( count )
> + {
> + writeb_relaxed(*(u8 *)from, to);
> + from++;
> + to++;
> + count--;
> + }
> +}
I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
we use a simple memcpy?
> +static inline int
> +shmem_channel_is_free(const volatile struct scmi_shared_mem __iomem *shmem)
> +{
> + return (readl(&shmem->channel_status) &
> + SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE) ? 0 : -EBUSY;
> +}
> +
> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
> + scmi_msg_header_t *hdr, void *data, int len)
> +{
> + int ret;
> +
> + if ( (len + sizeof(shmem->msg_header)) > SCMI_SHMEM_MAPPED_SIZE )
> + {
> + printk(XENLOG_ERR "scmi: Wrong size of smc message. Data is invalid\n");
> + return -EINVAL;
> + }
> +
> + ret = shmem_channel_is_free(shmem);
> + if ( ret )
> + return ret;
> +
> + writel_relaxed(0x0, &shmem->channel_status);
> + /* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
> + writel_relaxed(0x0, &shmem->flags);
> + writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
> + writel(pack_scmi_header(hdr), &shmem->msg_header);
> +
> + if ( len > 0 && data )
> + __memcpy_toio(shmem->msg_payload, data, len);
> +
> + return 0;
> +}
> +
> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
> + scmi_msg_header_t *hdr, void *data, int len)
> +{
> + int recv_len;
> + int ret;
> + int pad = sizeof(hdr->status);
> +
> + if ( len >= SCMI_SHMEM_MAPPED_SIZE - sizeof(shmem) )
> + {
> + printk(XENLOG_ERR
> + "scmi: Wrong size of input smc message. Data may be invalid\n");
> + return -EINVAL;
> + }
> +
> + ret = shmem_channel_is_free(shmem);
> + if ( ret )
> + return ret;
> +
> + recv_len = readl(&shmem->length) - sizeof(shmem->msg_header);
> +
> + if ( recv_len < 0 )
> + {
> + printk(XENLOG_ERR
> + "scmi: Wrong size of smc message. Data may be invalid\n");
> + return -EINVAL;
> + }
> +
> + unpack_scmi_header(readl(&shmem->msg_header), hdr);
> +
> + hdr->status = readl(&shmem->msg_payload);
> + recv_len = recv_len > pad ? recv_len - pad : 0;
> +
> + ret = scmi_to_xen_errno(hdr->status);
> + if ( ret )
> + {
> + printk(XENLOG_DEBUG "scmi: Error received: %d\n", ret);
> + return ret;
> + }
> +
> + if ( recv_len > len )
> + {
> + printk(XENLOG_ERR
> + "scmi: Not enough buffer for message %d, expecting %d\n",
> + recv_len, len);
> + return -EINVAL;
> + }
> +
> + if ( recv_len > 0 )
> + __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
> +
> + return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/firmware/scmi-shmem.h b/xen/arch/arm/firmware/scmi-shmem.h
> new file mode 100644
> index 0000000000..2f8e23ff76
> --- /dev/null
> +++ b/xen/arch/arm/firmware/scmi-shmem.h
> @@ -0,0 +1,45 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Arm System Control and Management Interface definitions
> + * Version 3.0 (DEN0056C)
> + * Shared Memory based Transport
> + *
> + * Copyright (c) 2024 EPAM Systems
> + */
> +
> +#ifndef XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
> +#define XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
NIT: ARM_FIRMWARE_SCMI_SHMEM_H
> +#include <xen/stdint.h>
> +
> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE BIT(0, UL)
> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_ERROR BIT(1, UL)
> +
> +struct scmi_shared_mem {
> + uint32_t reserved;
> + uint32_t channel_status;
> + uint32_t reserved1[2];
> + uint32_t flags;
> + uint32_t length;
> + uint32_t msg_header;
> + uint8_t msg_payload[];
> +};
> +
> +#define SCMI_SHMEM_MAPPED_SIZE PAGE_SIZE
> +
> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
> + scmi_msg_header_t *hdr, void *data, int len);
> +
> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
> + scmi_msg_header_t *hdr, void *data, int len);
> +#endif /* XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/firmware/scmi-smc-multiagent.c b/xen/arch/arm/firmware/scmi-smc-multiagent.c
> new file mode 100644
> index 0000000000..e023bca3a1
> --- /dev/null
> +++ b/xen/arch/arm/firmware/scmi-smc-multiagent.c
> @@ -0,0 +1,860 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
> + *
> + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> + * Copyright (c) 2025 EPAM Systems
> + */
> +
> +#include <xen/acpi.h>
> +
> +#include <xen/device_tree.h>
> +#include <xen/init.h>
> +#include <xen/iocap.h>
> +#include <xen/err.h>
> +#include <xen/libfdt/libfdt.h>
> +#include <xen/param.h>
> +#include <xen/sched.h>
> +#include <xen/vmap.h>
> +
> +#include <asm/firmware/sci.h>
> +#include <asm/smccc.h>
> +
> +#include "scmi-proto.h"
> +#include "scmi-shmem.h"
> +
> +#define SCMI_AGENT_ID_INVALID 0xFF
> +
> +static uint8_t __initdata opt_dom0_scmi_agent_id = SCMI_AGENT_ID_INVALID;
> +integer_param("dom0_scmi_agent_id", opt_dom0_scmi_agent_id);
> +
> +#define SCMI_SECONDARY_AGENTS "xen,scmi-secondary-agents"
> +
> +#define HYP_CHANNEL 0x0
> +
> +struct scmi_channel {
> + uint32_t agent_id;
> + uint32_t func_id;
> + domid_t domain_id;
> + uint64_t paddr;
> + uint64_t len;
> + struct scmi_shared_mem __iomem *shmem;
> + spinlock_t lock;
> + struct list_head list;
> +};
> +
> +struct scmi_data {
> + struct list_head channel_list;
> + spinlock_t channel_list_lock;
> + uint32_t func_id;
> + bool initialized;
> + uint32_t shmem_phandle;
> + struct dt_device_node *dt_dev;
> +};
> +
> +static struct scmi_data scmi_data;
> +
> +static int send_smc_message(struct scmi_channel *chan_info,
> + scmi_msg_header_t *hdr, void *data, int len)
> +{
> + struct arm_smccc_res resp;
> + int ret;
> +
> + ret = shmem_put_message(chan_info->shmem, hdr, data, len);
> + if ( ret )
> + return ret;
> +
> + arm_smccc_1_1_smc(chan_info->func_id, 0, 0, 0, 0, 0, 0, 0, &resp);
> +
> + if ( resp.a0 )
> + return -EOPNOTSUPP;
Why if repo.a0 != 0 then we assume -EOPNOTSUPP? Is this part of the SCMI
specification?
> + return 0;
> +}
> +
> +static int do_smc_xfer(struct scmi_channel *chan_info, scmi_msg_header_t *hdr,
> + void *tx_data, int tx_size, void *rx_data, int rx_size)
> +{
> + int ret = 0;
> +
> + ASSERT(chan_info && chan_info->shmem);
> +
> + if ( !hdr )
> + return -EINVAL;
> +
> + spin_lock(&chan_info->lock);
> +
> + printk(XENLOG_DEBUG
> + "scmi: agent_id = %d msg_id = %x type = %d, proto = %x\n",
> + chan_info->agent_id, hdr->id, hdr->type, hdr->protocol);
> +
> + ret = send_smc_message(chan_info, hdr, tx_data, tx_size);
> + if ( ret )
> + goto clean;
> +
> + ret = shmem_get_response(chan_info->shmem, hdr, rx_data, rx_size);
> +
> +clean:
> + printk(XENLOG_DEBUG
> + "scmi: get smc response agent_id = %d msg_id = %x proto = %x res=%d\n",
> + chan_info->agent_id, hdr->id, hdr->protocol, ret);
> +
> + spin_unlock(&chan_info->lock);
> +
> + return ret;
> +}
> +
> +static struct scmi_channel *get_channel_by_id(uint32_t agent_id)
> +{
> + struct scmi_channel *curr;
> + bool found = false;
> +
> + spin_lock(&scmi_data.channel_list_lock);
> + list_for_each_entry(curr, &scmi_data.channel_list, list)
> + {
> + if ( curr->agent_id == agent_id )
> + {
> + found = true;
> + break;
> + }
> + }
> +
> + spin_unlock(&scmi_data.channel_list_lock);
> + if ( found )
> + return curr;
> +
> + return NULL;
> +}
> +
> +static struct scmi_channel *acquire_scmi_channel(struct domain *d,
> + uint32_t agent_id)
> +{
> + struct scmi_channel *curr;
> + struct scmi_channel *ret = ERR_PTR(-ENOENT);
> +
> + spin_lock(&scmi_data.channel_list_lock);
> + list_for_each_entry(curr, &scmi_data.channel_list, list)
> + {
> + if ( curr->agent_id == agent_id )
> + {
> + if ( curr->domain_id != DOMID_INVALID )
> + {
> + ret = ERR_PTR(-EEXIST);
> + break;
> + }
> +
> + curr->domain_id = d->domain_id;
> + ret = curr;
> + break;
> + }
> + }
> +
> + spin_unlock(&scmi_data.channel_list_lock);
> +
> + return ret;
> +}
> +
> +static void relinquish_scmi_channel(struct scmi_channel *channel)
> +{
> + ASSERT(channel != NULL);
> +
> + spin_lock(&scmi_data.channel_list_lock);
> + channel->domain_id = DOMID_INVALID;
> + spin_unlock(&scmi_data.channel_list_lock);
> +}
> +
> +static int map_channel_memory(struct scmi_channel *channel)
> +{
> + ASSERT(channel && channel->paddr);
> + channel->shmem = ioremap_nocache(channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
ioremap is for MMIO, if these shared memory channels are on DDR, then it
would not be the right call. Are the "arm,scmi-shmem" address ranges
part of the memory node ranges? Or are they completely separate?
Also, why nocache? Wouldn't we want ioremap_cache?
> + if ( !channel->shmem )
> + return -ENOMEM;
> +
> + channel->shmem->channel_status = SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE;
> + printk(XENLOG_DEBUG "scmi: Got shmem %lx after vmap %p\n", channel->paddr,
> + channel->shmem);
> +
> + return 0;
> +}
> +
> +static void unmap_channel_memory(struct scmi_channel *channel)
> +{
> + ASSERT(channel && channel->shmem);
> + iounmap(channel->shmem);
> + channel->shmem = NULL;
> +}
> +
> +static struct scmi_channel *smc_create_channel(uint32_t agent_id,
> + uint32_t func_id, uint64_t addr)
> +{
> + struct scmi_channel *channel;
> +
> + channel = get_channel_by_id(agent_id);
> + if ( channel )
> + return ERR_PTR(EEXIST);
> +
> + channel = xmalloc(struct scmi_channel);
> + if ( !channel )
> + return ERR_PTR(ENOMEM);
> +
> + spin_lock_init(&channel->lock);
> + channel->agent_id = agent_id;
> + channel->func_id = func_id;
> + channel->domain_id = DOMID_INVALID;
> + channel->shmem = NULL;
> + channel->paddr = addr;
> + list_add_tail(&channel->list, &scmi_data.channel_list);
> + return channel;
> +}
> +
> +static void free_channel_list(void)
> +{
> + struct scmi_channel *curr, *_curr;
> +
> + list_for_each_entry_safe(curr, _curr, &scmi_data.channel_list, list)
> + {
> + list_del(&curr->list);
> + xfree(curr);
> + }
> +}
> +
> +static int __init
> +scmi_dt_read_hyp_channel_addr(struct dt_device_node *scmi_node, u64 *addr,
> + u64 *size)
> +{
> + struct dt_device_node *shmem_node;
> + const __be32 *prop;
> +
> + prop = dt_get_property(scmi_node, "shmem", NULL);
> + if ( !prop )
> + return -EINVAL;
> +
> + shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
> + if ( IS_ERR_OR_NULL(shmem_node) )
> + {
> + printk(XENLOG_ERR
> + "scmi: Device tree error, can't parse reserved memory %ld\n",
> + PTR_ERR(shmem_node));
> + return PTR_ERR(shmem_node);
> + }
> +
> + return dt_device_get_address(shmem_node, 0, addr, size);
> +}
> +
> +/*
> + * Handle Dom0 SCMI specific DT nodes
> + *
> + * Make a decision on copying SCMI specific nodes into Dom0 device tree.
> + * For SCMI multi-agent case:
> + * - shmem nodes will not be copied and generated instead if SCMI
> + * is enabled for Dom0
> + * - scmi node will be copied if SCMI is enabled for Dom0
> + */
> +static bool scmi_dt_handle_node(struct domain *d, struct dt_device_node *node)
> +{
> + static const struct dt_device_match skip_matches[] __initconst = {
> + DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
> + { /* sentinel */ },
> + };
> + static const struct dt_device_match scmi_matches[] __initconst = {
> + DT_MATCH_PATH("/firmware/scmi"),
> + { /* sentinel */ },
> + };
> +
> + if ( !scmi_data.initialized )
> + return false;
> +
> + /* always drop shmem */
> + if ( dt_match_node(skip_matches, node) )
> + {
> + dt_dprintk(" Skip scmi shmem\n");
> + return true;
> + }
> +
> + /* drop scmi if not enabled */
> + if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
> + {
> + dt_dprintk(" Skip scmi node\n");
> + return true;
> + }
> +
> + return false;
> +}
> +
> +/*
> + * Finalize Dom0 SCMI specific DT nodes
> + *
> + * if SCMI is enabled for Dom0:
> + * - generate shmem node
> + * - map SCMI shmem MMIO into Dom0
> + */
> +static int scmi_dt_finalize(struct domain *d, void *fdt)
> +{
> + __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
> + struct scmi_channel *channel;
> + int nodeoffset;
> + __be32 *cells;
> + __be32 val;
> + char buf[64];
> + int res, rc;
> +
> + if ( !sci_domain_is_enabled(d) )
> + return 0;
> +
> + channel = d->arch.sci_data;
> +
> + /*
> + * Replace "arm,smc-id" with proper value assigned for Dom0 SCMI channel
> + */
> + nodeoffset = fdt_node_offset_by_compatible(fdt, -1, "arm,scmi-smc");
> + if ( nodeoffset < 0 )
> + return -ENODEV;
> +
> + cells = (__be32 *)&val;
> + dt_set_cell(&cells, 1, channel->func_id);
> + res = fdt_setprop_inplace(fdt, nodeoffset, "arm,smc-id", &val, sizeof(val));
> + if ( res )
> + return -EINVAL;
> +
Are you sure it is worth to go through all this trouble to modify FDT in
place when we could simply generate the DT node from scratch like we do
for example for the GIC? This seems to be more error prone as well. Is
generating it from scratch is really difficult? If it is difficult then OK.
> + /*
> + * All SCMI shmem nodes should be removed from Dom0 DT at this point, so
> + * the shmem node for Dom0 need to be generated from SCMI channel assigned
> + * to Dom0.
> + * The original SCMI shmem node from platform DT is used by Xen SCMI driver
> + * itself as privileged channel (agent_id=0) to manage other SCMI
> + * agents (domains).
> + */
> + snprintf(buf, sizeof(buf), "scmi-shmem@%lx", channel->paddr);
> +
> + res = fdt_begin_node(fdt, buf);
> + if ( res )
> + return res;
> +
> + res = fdt_property_string(fdt, "compatible", "arm,scmi-shmem");
> + if ( res )
> + return res;
> +
> + cells = ®[0];
> +
> + dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> + channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
> +
> + res = fdt_property(fdt, "reg", reg, sizeof(reg));
> + if ( res )
> + return res;
> +
> + res = fdt_property_cell(fdt, "phandle", scmi_data.shmem_phandle);
> + if ( res )
> + return res;
> +
> + res = fdt_end_node(fdt);
> + if ( res )
> + return res;
> +
> + /*
> + * Map SCMI shmem into Dom0 here as shmem nodes are excluded from
> + * generic Dom0 DT processing
> + */
> + res = iomem_permit_access(d, paddr_to_pfn(channel->paddr),
> + paddr_to_pfn(channel->paddr +
> + SCMI_SHMEM_MAPPED_SIZE - 1));
> + if ( res )
> + return res;
> +
> + res = map_regions_p2mt(d, gaddr_to_gfn(channel->paddr),
> + PFN_UP(SCMI_SHMEM_MAPPED_SIZE),
> + maddr_to_mfn(channel->paddr), p2m_mmio_direct_nc);
> + if ( res )
> + {
> + rc = iomem_deny_access(d, paddr_to_pfn(channel->paddr),
> + paddr_to_pfn(channel->paddr +
> + SCMI_SHMEM_MAPPED_SIZE - 1));
> + if ( rc )
> + printk(XENLOG_ERR "scmi: Unable to deny iomem access , err = %d\n",
> + rc);
> + }
> +
> + return res;
> +}
> +
> +static int scmi_assign_device(uint32_t agent_id, uint32_t device_id,
> + uint32_t flags)
> +{
> + struct scmi_msg_base_set_device_permissions_a2p tx;
> + struct scmi_channel *channel;
> + scmi_msg_header_t hdr;
> + int ret;
> +
> + channel = get_channel_by_id(HYP_CHANNEL);
> + if ( !channel )
> + return -EINVAL;
> +
> + hdr.id = SCMI_BASE_SET_DEVICE_PERMISSIONS;
> + hdr.type = 0;
> + hdr.protocol = SCMI_BASE_PROTOCOL;
> +
> + tx.agent_id = agent_id;
> + tx.device_id = device_id;
> + tx.flags = flags;
> +
> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
> + if ( ret == -EOPNOTSUPP )
> + return 0;
Is it actually OK to pretend that everything worked if the return is
-EOPNOTSUPP? I mean that in this case can we assume that the device is
actually assigned anyway? Wouldn't follow up SCMI operations on this
device fail?
> + return ret;
> +}
> +
> +static int scmi_dt_assign_device(struct domain *d,
> + struct dt_phandle_args *ac_spec)
> +{
> + struct scmi_channel *agent_channel;
> + uint32_t scmi_device_id = ac_spec->args[0];
> + int ret;
> +
> + if ( !d->arch.sci_data )
> + return 0;
> +
> + /* The access-controllers is specified for DT dev, but it's not a SCMI */
> + if ( ac_spec->np != scmi_data.dt_dev )
> + return 0;
I wonder if this should be an error
> + agent_channel = d->arch.sci_data;
> +
> + spin_lock(&agent_channel->lock);
> +
> + ret = scmi_assign_device(agent_channel->agent_id, scmi_device_id,
> + SCMI_BASE_DEVICE_ACCESS_ALLOW);
> + if ( ret )
> + {
> + printk(XENLOG_ERR
> + "scmi: could not assign dev for %pd agent:%d dev_id:%u (%d)",
> + d, agent_channel->agent_id, scmi_device_id, ret);
> + }
> +
> + spin_unlock(&agent_channel->lock);
> + return ret;
> +}
> +
> +static __init int collect_agents(struct dt_device_node *scmi_node)
> +{
> + const struct dt_device_node *chosen_node;
> + const __be32 *prop;
> + uint32_t len, i;
> +
> + chosen_node = dt_find_node_by_path("/chosen");
> + if ( !chosen_node )
> + {
> + printk(XENLOG_ERR "scmi: chosen node not found\n");
> + return -ENOENT;
> + }
> +
> + prop = dt_get_property(chosen_node, SCMI_SECONDARY_AGENTS, &len);
> + if ( !prop )
> + {
> + printk(XENLOG_WARNING "scmi: No %s property found\n",
> + SCMI_SECONDARY_AGENTS);
> + return -ENODEV;
> + }
> +
> + if ( len % (3 * sizeof(uint32_t)) )
> + {
> + printk(XENLOG_ERR "scmi: Invalid length of %s property: %d\n",
> + SCMI_SECONDARY_AGENTS, len);
> + return -EINVAL;
> + }
> +
> + for ( i = 0; i < len / (3 * sizeof(uint32_t)); i++ )
> + {
> + uint32_t agent_id = be32_to_cpu(*prop++);
> + uint32_t smc_id = be32_to_cpu(*prop++);
> + uint32_t shmem_phandle = be32_to_cpu(*prop++);
> + struct dt_device_node *node = dt_find_node_by_phandle(shmem_phandle);
> + u64 addr, size;
> + int ret;
> +
> + if ( !node )
> + {
> + printk(XENLOG_ERR "scmi: Could not find shmem node for agent %d\n",
> + agent_id);
> + return -EINVAL;
> + }
> +
> + ret = dt_device_get_address(node, 0, &addr, &size);
> + if ( ret )
> + {
> + printk(XENLOG_ERR
> + "scmi: Could not read shmem address for agent %d: %d",
> + agent_id, ret);
> + return ret;
> + }
> +
> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
> + {
> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
> + return -EINVAL;
> + }
> +
> + ret = PTR_RET(smc_create_channel(agent_id, smc_id, addr));
> + if ( ret )
> + {
> + printk(XENLOG_ERR "scmi: Could not create channel for agent %d: %d",
> + agent_id, ret);
> + return ret;
> + }
> +
> + printk(XENLOG_DEBUG "scmi: Agent %d SMC %X addr %lx\n", agent_id,
> + smc_id, addr);
> + }
> +
> + return 0;
> +}
> +
> +static int scmi_domain_init(struct domain *d,
> + struct xen_domctl_createdomain *config)
> +{
> + struct scmi_channel *channel;
> + int ret;
> +
> + if ( !scmi_data.initialized )
> + return 0;
> +
> + /*
> + * Special case for Dom0 - the SCMI support is enabled basing on
> + * "dom0_sci_agent_id" Xen command line parameter
> + */
> + if ( is_hardware_domain(d) )
> + {
> + if ( opt_dom0_scmi_agent_id != SCMI_AGENT_ID_INVALID )
> + {
> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> + config->arch.arm_sci_agent_id = opt_dom0_scmi_agent_id;
> + }
> + else
> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> + }
> +
> + if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE )
> + return 0;
> +
> + channel = acquire_scmi_channel(d, config->arch.arm_sci_agent_id);
> + if ( IS_ERR(channel) )
> + {
> + printk(XENLOG_ERR
> + "scmi: Failed to acquire SCMI channel for agent_id %u: %ld\n",
> + config->arch.arm_sci_agent_id, PTR_ERR(channel));
> + return PTR_ERR(channel);
> + }
> +
> + printk(XENLOG_INFO
> + "scmi: Acquire channel id = 0x%x, domain_id = %d paddr = 0x%lx\n",
> + channel->agent_id, channel->domain_id, channel->paddr);
> +
> + /*
> + * Dom0 (if present) needs to have an access to the guest memory range
> + * to satisfy iomem_access_permitted() check in XEN_DOMCTL_iomem_permission
> + * domctl.
Ideally this should not be needed but I understand we don't have an
easy solution, I think we can go ahead with this for now.
> + */
> + if ( hardware_domain && !is_hardware_domain(d) )
> + {
> + ret = iomem_permit_access(hardware_domain, paddr_to_pfn(channel->paddr),
> + paddr_to_pfn(channel->paddr + PAGE_SIZE - 1));
> + if ( ret )
> + goto error;
> + }
> +
> + d->arch.sci_data = channel;
> + d->arch.sci_enabled = true;
> +
> + return 0;
> +
> +error:
> + relinquish_scmi_channel(channel);
> + return ret;
> +}
> +
> +int scmi_domain_sanitise_config(struct xen_domctl_createdomain *config)
> +{
> + if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
> + config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA )
> + {
> + dprintk(XENLOG_INFO, "scmi: Unsupported ARM_SCI type\n");
> + return -EINVAL;
> + }
> + else if ( config->arch.arm_sci_type ==
> + XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA &&
> + config->arch.arm_sci_agent_id == 0 )
> + {
> + dprintk(XENLOG_INFO,
> + "scmi: A zero ARM SCMI agent_id is not supported\n");
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static int scmi_relinquish_resources(struct domain *d)
> +{
> + int ret;
> + struct scmi_channel *channel, *agent_channel;
> + scmi_msg_header_t hdr;
> + struct scmi_msg_base_reset_agent_cfg_a2p tx;
> +
> + if ( !d->arch.sci_data )
> + return 0;
> +
> + agent_channel = d->arch.sci_data;
> +
> + spin_lock(&agent_channel->lock);
> + tx.agent_id = agent_channel->agent_id;
> + spin_unlock(&agent_channel->lock);
> +
> + channel = get_channel_by_id(HYP_CHANNEL);
> + if ( !channel )
> + {
> + printk(XENLOG_ERR
> + "scmi: Unable to get Hypervisor scmi channel for domain %d\n",
> + d->domain_id);
> + return -EINVAL;
> + }
> +
> + hdr.id = SCMI_BASE_RESET_AGENT_CONFIGURATION;
> + hdr.type = 0;
> + hdr.protocol = SCMI_BASE_PROTOCOL;
> +
> + tx.flags = 0;
> +
> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
> + if ( ret == -EOPNOTSUPP )
> + return 0;
> +
> + return ret;
> +}
> +
> +static void scmi_domain_destroy(struct domain *d)
> +{
> + struct scmi_channel *channel;
> +
> + if ( !d->arch.sci_data )
> + return;
> +
> + channel = d->arch.sci_data;
> + spin_lock(&channel->lock);
> +
> + relinquish_scmi_channel(channel);
> + printk(XENLOG_DEBUG "scmi: Free domain %d\n", d->domain_id);
> +
> + d->arch.sci_data = NULL;
> + d->arch.sci_enabled = true;
> +
> + spin_unlock(&channel->lock);
> +}
> +
> +static bool scmi_handle_call(struct cpu_user_regs *regs)
> +{
> + uint32_t fid = (uint32_t)get_user_reg(regs, 0);
> + struct scmi_channel *agent_channel;
> + struct domain *d = current->domain;
> + struct arm_smccc_res resp;
> + bool res = false;
> +
> + if ( !sci_domain_is_enabled(d) )
> + return false;
> +
> + agent_channel = d->arch.sci_data;
> + spin_lock(&agent_channel->lock);
> +
> + if ( agent_channel->func_id != fid )
> + {
> + res = false;
> + goto unlock;
> + }
> +
> + arm_smccc_1_1_smc(fid,
> + get_user_reg(regs, 1),
> + get_user_reg(regs, 2),
> + get_user_reg(regs, 3),
> + get_user_reg(regs, 4),
> + get_user_reg(regs, 5),
> + get_user_reg(regs, 6),
> + get_user_reg(regs, 7),
> + &resp);
> +
> + set_user_reg(regs, 0, resp.a0);
> + set_user_reg(regs, 1, resp.a1);
> + set_user_reg(regs, 2, resp.a2);
> + set_user_reg(regs, 3, resp.a3);
> + res = true;
> +unlock:
> + spin_unlock(&agent_channel->lock);
> +
> + return res;
> +}
> +
> +static const struct sci_mediator_ops scmi_ops = {
> + .domain_init = scmi_domain_init,
> + .domain_destroy = scmi_domain_destroy,
> + .relinquish_resources = scmi_relinquish_resources,
> + .handle_call = scmi_handle_call,
> + .dom0_dt_handle_node = scmi_dt_handle_node,
> + .dom0_dt_finalize = scmi_dt_finalize,
> + .domain_sanitise_config = scmi_domain_sanitise_config,
> + .assign_dt_device = scmi_dt_assign_device,
> +};
> +
> +static int __init scmi_check_smccc_ver(void)
> +{
> + if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
> + {
> + printk(XENLOG_WARNING
> + "scmi: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
> + return -ENOSYS;
> + }
> +
> + return 0;
> +}
> +
> +static __init int scmi_probe(struct dt_device_node *scmi_node, const void *data)
> +{
> + u64 addr, size;
> + int ret, i;
> + struct scmi_channel *channel, *agent_channel;
> + int n_agents;
> + scmi_msg_header_t hdr;
> + struct scmi_msg_base_attributes_p2a rx;
> +
> + ASSERT(scmi_node != NULL);
> +
> + INIT_LIST_HEAD(&scmi_data.channel_list);
> + spin_lock_init(&scmi_data.channel_list_lock);
> +
> + if ( !acpi_disabled )
> + {
> + printk(XENLOG_WARNING "scmi: is not supported when using ACPI\n");
> + return -EINVAL;
> + }
> +
> + ret = scmi_check_smccc_ver();
> + if ( ret )
> + return ret;
> +
> + if ( !dt_property_read_u32(scmi_node, "arm,smc-id", &scmi_data.func_id) )
> + {
> + printk(XENLOG_ERR "scmi: unable to read smc-id from DT\n");
> + return -ENOENT;
> + }
> +
> + /* save shmem phandle and re-use it fro Dom0 DT shmem node */
> + if ( !dt_property_read_u32(scmi_node, "shmem", &scmi_data.shmem_phandle) )
> + {
> + printk(XENLOG_ERR "scmi: unable to read shmem phandle from DT\n");
> + return -ENOENT;
> + }
> +
> + ret = scmi_dt_read_hyp_channel_addr(scmi_node, &addr, &size);
> + if ( IS_ERR_VALUE(ret) )
> + return -ENOENT;
> +
> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
> + {
> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
> + return -EINVAL;
> + }
> +
> + scmi_data.dt_dev = scmi_node;
> +
> + channel = smc_create_channel(HYP_CHANNEL, scmi_data.func_id, addr);
> + if ( IS_ERR(channel) )
> + goto out;
> +
> + ret = map_channel_memory(channel);
> + if ( ret )
> + goto out;
> +
> + channel->domain_id = DOMID_XEN;
> +
> + hdr.id = SCMI_BASE_PROTOCOL_ATTIBUTES;
> + hdr.type = 0;
> + hdr.protocol = SCMI_BASE_PROTOCOL;
> +
> + ret = do_smc_xfer(channel, &hdr, NULL, 0, &rx, sizeof(rx));
> + if ( ret )
> + goto error;
> +
> + n_agents = SCMI_FIELD_GET(SCMI_BASE_ATTR_NUM_AGENT, rx.attributes);
> + printk(XENLOG_DEBUG "scmi: Got agent count %d\n", n_agents);
> +
> + ret = collect_agents(scmi_node);
> + if ( ret )
> + goto error;
> +
> + i = 1;
> +
> + list_for_each_entry(agent_channel, &scmi_data.channel_list, list)
> + {
> + struct scmi_msg_base_discover_agent_p2a da_rx;
> + struct scmi_msg_base_discover_agent_a2p da_tx;
> +
> + ret = map_channel_memory(agent_channel);
> + if ( ret )
> + goto error;
> +
> + hdr.id = SCMI_BASE_DISCOVER_AGENT;
> + hdr.type = 0;
> + hdr.protocol = SCMI_BASE_PROTOCOL;
> +
> + da_tx.agent_id = agent_channel->agent_id;
> +
> + ret = do_smc_xfer(agent_channel, &hdr, &da_tx, sizeof(da_tx), &da_rx,
> + sizeof(da_rx));
> + if ( agent_channel->domain_id != DOMID_XEN )
> + unmap_channel_memory(agent_channel);
> + if ( ret )
> + goto error;
> +
> + printk(XENLOG_DEBUG "id=0x%x name=%s\n", da_rx.agent_id, da_rx.name);
> +
> + agent_channel->agent_id = da_rx.agent_id;
It is OK to set agent_channel->agent_id to the value provided by the
SCMI server, but if we are also taking the agent_channel->agent_id value
from the user via device tree, shouldn't we throw an error if there is a
mismatch?
Or even better: can we avoid taking the value via device tree to make it
easier to configure?
> + if ( i > n_agents )
> + break;
> +
> + i++;
> + }
> +
> + ret = sci_register(&scmi_ops);
> + if ( ret )
> + {
> + printk(XENLOG_ERR "SCMI: mediator already registered (ret = %d)\n",
> + ret);
> + return ret;
> + }
> +
> + scmi_data.initialized = true;
> + goto out;
> +
> +error:
> + unmap_channel_memory(channel);
> + free_channel_list();
> +out:
> + return ret;
> +}
> +
> +static const struct dt_device_match scmi_smc_match[] __initconst = {
> + DT_MATCH_COMPATIBLE("arm,scmi-smc"),
> + { /* sentinel */ },
> +};
> +
> +DT_DEVICE_START(scmi_smc_ma, "SCMI SMC MEDIATOR", DEVICE_FIRMWARE)
> + .dt_match = scmi_smc_match,
> + .init = scmi_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 095b1a23e3..30e46de6d7 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -329,6 +329,7 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>
> #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
> #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC 1
> +#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA 2
>
> struct xen_arch_domainconfig {
> /* IN/OUT */
> @@ -355,6 +356,8 @@ struct xen_arch_domainconfig {
> uint32_t clock_frequency;
> /* IN */
> uint8_t arm_sci_type;
> + /* IN */
> + uint8_t arm_sci_agent_id;
> };
> #endif /* __XEN__ || __XEN_TOOLS__ */
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent
2025-05-19 15:50 ` [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent Oleksii Moisieiev
@ 2025-05-23 20:19 ` Stefano Stabellini
2025-06-12 12:03 ` Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Stefano Stabellini @ 2025-05-23 20:19 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Stefano Stabellini,
Volodymyr Babchuk, Grygorii Strashko
[-- Attachment #1: Type: text/plain, Size: 11964 bytes --]
On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
>
> Proposal description to add separate SCMI DT node for Xen management agent
> under "chosen" or xen-config node, like Hyperlaunch "xen,config".
I think it is OK to place a larger "xen,config" node under /chosen with
more information for Xen to setup SCMI more easily.
> This proposal introduces a new approach to the Xen multi-domain
> configuration, where all Xen-specific configuration has been moved
> under the "/chosen" node. This requires less Dom0 device tree
> manipulation and isolates Xen configuration from domain configuration.
>
> This approach provides the following device tree (DT) parameters:
>
> - "xen,scmi-secondary-agents": A Xen-specific parameter under the
> "/chosen" node, which describes the SCMI agent configuration for
> the domains.
> - the SCMI configuration for Xen (privileged agent) and the shared
> memory configuration for all agents are provided under the "/chosen"
> node and are used strictly by Xen for its initial configuration.
> - the scmi_shm and SCMI configuration for Dom0 are placed in the
> "/firmware/scmi" node so that they can be moved to Dom0 without
> any changes.
Isn't the SCMI configuration present in /firmware/scmi referring to the
privileged agent=0 meant to be used by Xen?
I certainly see benefits in simplifying the configuration and especially
reducing the amount of changes a user might have to make on the
underlying device tree, but if the user needs to change /firmware/scmi
with the Dom0 information, it seems more dangerous and error prone than
the previous approach.
> This configuration allows the use of Xen-specific nodes to provide
> information strictly needed by Xen while using the default SCMI
> configuration for Dom0 and other domains. As a result, no additional
> bindings need to be introduced to the device tree.
This is not actually implemented by this patch series, right?
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> ---
>
>
>
> .../arm/firmware/arm-scmi-proposal.rst | 224 ++++++++++++++++++
> 1 file changed, 224 insertions(+)
> create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
>
> diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> new file mode 100644
> index 0000000000..fcc2ed2b65
> --- /dev/null
> +++ b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> @@ -0,0 +1,224 @@
> +
> +Proposal for SCMI multi-agent driver bindings
> +=============================================
> +
> +Now the Xen configuration for SCMI multi-agent support is done in a bit complicated way, especially
> +from SCMI multi-agent driver initialization and Dom0 DT manipulation point of view.
> +Also it does not take into account future requirements to support SCP SCMI FW.
> +
> +To enable SCMI multi-agent user need:
> +
> +* take host DT with basic SCMI enabled
> +* add SCMI shared-memory nodes for all agents
> +* update SCMI node to point on SCMI Xen management channel (``[smc-id, shmem]``)
> +* add "xen,scmi-secondary-agents" property to the "\chosen" node
> +
> +.. code::
> +
> + chosen {
> + xen,scmi-secondary-agents = <
> + 1 0x82000003 &scmi_shm_1
> + 2 0x82000004 &scmi_shm_2
> + 3 0x82000005 &scmi_shm_3
> + 4 0x82000006 &scmi_shm_4>;
> + }
> +
> + /{
> + // SCMI shared-memory nodes for all agents
> + scmi_shm_0 : sram@47ff0000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> + };
> + scmi_shm_1: sram@47ff1000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> + };
> + scmi_shm_2: sram@47ff2000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> + };
> + scmi_shm_3: sram@47ff3000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> + };
> + scmi_shm_4: sram@47ff4000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> + };
> +
> + firmware {
> + scmi: scmi {
> + compatible = "arm,scmi-smc";
> + arm, smc - id = <0x82000002>; <--- Xen management agent channel "smc-id"
> + #address-cells = < 1>;
> + #size-cells = < 0>;
> + #access-controller-cells = < 1>;
> + shmem = <&scmi_shm_0>; <--- Xen management agent channel "shmem"
> +
> + protocol@X{
> + };
> + };
> + };
> + }
> +
> +Important thing to note is that all information about multi-channel support is strictly Xen specific.
> +
> +During initialization the SCMI multi-agent driver uses Host DT SCMI node and
> +"xen,scmi-secondary-agents" property to init itself and then, during Dom0 creation, manipulates
> +Dom0 DT to remove Xen specific SCMI info and update dom0 SCMI nodes with Dom0 SCMI agent specific
> +information.
> +
> +There are two negative points:
> +
> +1) Double DT modification - one is user to set up SCMI Xen support in Host DT, second -
> + Dom0 DT manipulation.
> +2) In case of future support of mailbox shared-memory transport there could be up to 4 mailboxes and
> + up to 2 shared-memories per SCMI agent channel.
> +
> +Hence SCMI multi-agent support is Xen specific knowledge there is a proposal to add it as Xen
> +specific DT definitions and so minimize Host and Dom0 DT manipulations.
> +Those definitions can be added in "/chosen" or, ideally, in "xen,config" node (like in Hyperlaunch design).
> +
> +The SCMI binding stays generic, just two SCMI nodes defined - one for Xen management channel and
> +one for Host Dom0 OSPM.
> +
> +Example of using "chosen" for configuration:
> +
> +.. code::
> +
> + /{
> +
> + chosen {
> + ...
> +
> + // Xen SCMI management channel
> + scmi_shm_0 : sram@47ff0000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> + };
> + scmi_xen: scmi {
> + compatible = "arm,scmi-smc";
> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
> + #address-cells = < 1>;
> + #size-cells = < 0>;
> + #access-controller-cells = < 1>;
> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
> + };
> +
> + // SCMI multi-agent configuration
> + scmi_shm_2: sram@47ff2000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> + };
> + scmi_shm_3: sram@47ff3000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> + };
> + scmi_shm_4: sram@47ff4000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> + };
> + xen,scmi-secondary-agents = <
> + 1 0x82000003 &scmi_shm
> + 2 0x82000004 &scmi_shm_2
> + 3 0x82000005 &scmi_shm_3
> + 4 0x82000006 &scmi_shm_4>;
> + };
> +
> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
> + scmi_shm: sram@47ff1000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> + };
> +
> + firmware {
> + scmi: scmi {
> + compatible = "arm,scmi-smc";
> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
> + #address-cells = < 1>;
> + #size-cells = < 0>;
> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
By OSPM you mean Dom0 and not Xen? So this is a change compared to a
device tree for baremetal Linux without Xen?
Let me ask the same question differently. In the case of barematal Linux
without Xen (no KVM), what would Linux see under /firmware/scmi as
smc-id and shmem? The same as the one that Xen would use for itself? Or
the same as the ones that Dom0 would use when Xen is present?
> + protocol@X{
> + };
> + };
> + };
> + }
> +
> +
> +In the above case:
> +
> +1) Xen SCMI multi-agent can be probed with DT configuration from "chosen" (or special "xen,config")
> + node and all Xen related nodes can be easily dropped from Dom0 DT.
> +2) Host SCMI OSPM channel DT nodes can be copied to Dom0 DT without changes if SCMI enabled for it.
> +3) Future support for mailbox shared-memory transport (SCP SCMI FW) can be simplified as no more
> + manipulation required with Dom0 SCMI "arm,smc-id" and "shmem" DT properties.
Yes, I can see the benefit if we can arrange it so that the underlying
host device tree is the same that Linux would use baremetal. And all the
extra configuration is placed under /chosen in "xen,config" node or
similar. I would probably call it "xen,scmi".
> +Example of using "xen,config" for configuration:
> +
> +.. code::
> +
> + hypervisor {
> + compatible = “hypervisor,xen”
> +
> + // Configuration container
> + config {
> + compatible = "xen,config";
> + ...
> +
> + // Xen SCMI management channel
> + scmi_shm_0 : sram@47ff0000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> + };
> + scmi_xen: scmi {
> + compatible = "arm,scmi-smc";
> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
> + #address-cells = < 1>;
> + #size-cells = < 0>;
> + #access-controller-cells = < 1>;
> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
> + };
> +
> + // SCMI multi-agent configuration
> + scmi_shm_2: sram@47ff2000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> + };
> + scmi_shm_3: sram@47ff3000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> + };
> + scmi_shm_4: sram@47ff4000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> + };
> + xen,scmi-secondary-agents = <
> + 1 0x82000003 &scmi_shm
> + 2 0x82000004 &scmi_shm_2
> + 3 0x82000005 &scmi_shm_3
> + 4 0x82000006 &scmi_shm_4>;
> + };
> + };
> +
> + /{
> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
> + scmi_shm: sram@47ff1000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> + };
> +
> + firmware {
> + scmi: scmi {
> + compatible = "arm,scmi-smc";
> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
> + #address-cells = < 1>;
> + #size-cells = < 0>;
> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
> +
> + protocol@X{
> + };
> + };
> + };
> + }
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-05-23 20:06 ` Stefano Stabellini
@ 2025-06-02 7:17 ` Bertrand Marquis
2025-06-12 12:03 ` Oleksii Moisieiev
2025-06-12 12:10 ` Grygorii Strashko
2025-06-05 22:44 ` Julien Grall
2025-06-12 12:03 ` Oleksii Moisieiev
2 siblings, 2 replies; 67+ messages in thread
From: Bertrand Marquis @ 2025-06-02 7:17 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Oleksii Moisieiev, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
Hi Stefano and Oleksii,
> On 23 May 2025, at 22:06, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> One question for Bertrand below
>
>
> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
>> below.
>>
>> +-----------------------------------------+
>> | |
>> | EL3 TF-A SCMI |
>> +-------+--+-------+--+-------+--+-------++
>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
>> +-----+-+ +---+---+ +--+----+ +---+---+
>> smc-id0 | | | |
>> agent0 | | | |
>> +-----v--------+---------+-----------+----+
>> | | | | |
>> | | | | |
>> +--------------+---------+-----------+----+
>> smc-id1 | smc-id2| smc-idX|
>> agent1 | agent2 | agentX |
>> | | |
>> +----v---+ +--v-----+ +--v-----+
>> | | | | | |
>> | Dom0 | | Dom1 | | DomX |
>> | | | | | |
>> | | | | | |
>> +--------+ +--------+ +--------+
>>
>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
>> memory transport for every Agent in the system.
>>
>> The SCMI Agent transport channel defined by pair:
>> - smc-id: SMC/HVC id used for Doorbell
>> - shmem: shared memory for messages transfer, Xen page aligned,
>> p2m_mmio_direct_nc.
>>
>> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
>> multi-agent functionality under Xen:
>> - Xen manegement agent: trusted agents that accesses to the Base Protocol
>> commands to configure agent specific permissions
>> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
>> allowed direct HW access. At least one OSPM VM agent has to be provided
>> by FW if HW is handled only by Dom0 or Driver Domain.
>>
>> The EL3 SCMI FW expected to implement following Base protocol messages:
>> - BASE_DISCOVER_AGENT
>> - BASE_RESET_AGENT_CONFIGURATION (optional)
>> - BASE_SET_DEVICE_PERMISSIONS (optional)
>>
>> The SCI SCMI SMC multi-agent driver implements following functionality:
>> - It's initialized based on the Host DT SCMI node (only one SCMI interface
>> is supported) which describes Xen management agent SCMI interface.
>>
>> scmi_shm_0 : sram@47ff0000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff0000 0x0 0x1000>;
>> };
>> firmware {
>> scmi: scmi {
>> compatible = "arm,scmi-smc";
>> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
>
> some extra spaces, it might be a copy/paste error
>
>
>> \#address-cells = < 1>;
>> \#size-cells = < 0>;
>> \#access-controller - cells = < 1>;
>> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
>>
>> protocol@X{
>> };
>> };
>> };
>>
>> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
>> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
>>
>> chosen {
>> xen,scmi-secondary-agents = <
>> 1 0x82000003 &scmi_shm_1
>> 2 0x82000004 &scmi_shm_2
>> 3 0x82000005 &scmi_shm_3
>> 4 0x82000006 &scmi_shm_4>;
>> }
>>
>> /{
>> scmi_shm_1: sram@47ff1000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff1000 0x0 0x1000>;
>> };
>> scmi_shm_2: sram@47ff2000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff2000 0x0 0x1000>;
>> };
>> scmi_shm_3: sram@47ff3000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff3000 0x0 0x1000>;
>> };
>> }
>> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
>> this agent_id.
>>
>> Note that Xen is the only one entry in the system which need to know
>> about SCMI multi-agent support.
>>
>> - It implements the SCI subsystem interface required for configuring and
>> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
>> SCMI functionality for domain it has to be configured with unique supported
>> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
>> [smc-id, shmem] defined for this SCMI Agent_id.
>> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
>> -- zero-copy, the guest domain puts SCMI message in shmem;
>> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
>> -- the Xen driver catches exception, do checks and synchronously forwards
>> it to EL3 FW.
>> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
>> management agent channel on domain destroy event. This allows to reset
>> resources used by domain and so implement use-case like domain reboot.
>>
>> Dom0 Enable SCMI SMC:
>> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
>> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
>> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
>> node according to assigned agent_id.
>>
>> Guest domains enable SCMI SMC:
>> - xl.cfg: add configuration option as below
>>
>> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
>>
>> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
>> the domain, for example:
>>
>> iomem = [
>> "47ff2,1@22001",
>> ]
>
> Looking at the code and the configuration options, it looks like it is
> possible to map a scmi-shmem channel at a different address for the
> guest. It seems like it would work. Is that correct?
>
>
>> - DT: add SCMI nodes to the Driver domain partial device tree as in the
>> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
>>
>> passthrough {
>> scmi_shm_0: sram@22001000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x22001000 0x0 0x1000>;
>> };
>>
>> firmware {
>> compatible = "simple-bus";
>> scmi: scmi {
>> compatible = "arm,scmi-smc";
>> arm,smc-id = <0x82000004>;
>> shmem = <&scmi_shm_0>;
>> ...
>> }
>> }
>> }
>>
>> SCMI "4.2.1.1 Device specific access control"
>>
>> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
>> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
>> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
>> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
>> to the Xen SCMI.
>>
>> &i2c1 {
>> access-controllers = <&scmi 0>;
>> };
>>
>> The Dom0 and dom0less domains DT devices will be processed automatically through
>> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
>> shell be used:
>>
>> dtdev = [
>> "/soc/i2c@e6508000",
>> ]
>>
>> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
>>
>> [1] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
>> [2] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
>> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>
> Thanks for the long explanation, great work! I am really looking forward
> to have this feature in the tree soon.
>
>
>> ---
>>
>> Changes in v4:
>> - toolstack comments from Anthony PERARD
>> - added dom0less support
>> - added doc for "xen,scmi-secondary-agents"
>>
>> docs/man/xl.cfg.5.pod.in | 13 +
>> docs/misc/arm/device-tree/booting.txt | 60 ++
>> docs/misc/xen-command-line.pandoc | 9 +
>> tools/libs/light/libxl_arm.c | 4 +
>> tools/libs/light/libxl_types.idl | 4 +-
>> tools/xl/xl_parse.c | 12 +
>> xen/arch/arm/dom0less-build.c | 11 +
>> xen/arch/arm/domain_build.c | 3 +-
>> xen/arch/arm/firmware/Kconfig | 11 +
>> xen/arch/arm/firmware/Makefile | 1 +
>> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
>> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
>> xen/arch/arm/firmware/scmi-shmem.h | 45 +
>> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
>> xen/include/public/arch-arm.h | 3 +
>> 15 files changed, 1371 insertions(+), 2 deletions(-)
>> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
>> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
>>
>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>> index 1ccf50b8ea..302c46d8bc 100644
>> --- a/docs/man/xl.cfg.5.pod.in
>> +++ b/docs/man/xl.cfg.5.pod.in
>> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
>> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
>> option.
>>
>> +=item B<scmi_smc_multiagent>
>> +
>> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
>> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
>> +specified for the guest.
>> +
>> =back
>>
>> +=item B<agent_id=NUMBER>
>> +
>> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
>> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
>> +existing on a single host must be unique and in the range [1..255].
>> +
>> =back
>>
>> =back
>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
>> index 8943c04173..c8923ab8b2 100644
>> --- a/docs/misc/arm/device-tree/booting.txt
>> +++ b/docs/misc/arm/device-tree/booting.txt
>> @@ -296,6 +296,20 @@ with the following properties:
>> Should be used together with dom0_scmi_smc_passthrough Xen command line
>> option.
>>
>> + - "scmi_smc_multiagent"
>> +
>> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>> + SMC calls forwarding from domain to the EL3 firmware (like ARM
>> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
>> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
>> + property.
>> +
>> +- "xen,sci_agent_id"
>> +
>> + Specifies a non-zero ARM SCI agent id for the guest. This option is
>> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
>> + the guest. The agent ids of guest must be unique and in the range [1..255].
>> +
>> Under the "xen,domain" compatible node, one or more sub-nodes are present
>> for the DomU kernel and ramdisk.
>>
>> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
>> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
>> guest physical address space. DomU1 is explicitly defined as the owner domain,
>> and DomU2 is the borrower domain.
>> +
>> +SCMI SMC multi-agent support
>> +============================
>> +
>> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
>> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
>> +according to the SCMI compliant EL3 Firmware specification with
>> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
>> +the top-level "chosen" node:
>> +
>> +- xen,scmi-secondary-agents
>> +
>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
>> + available for Xen. Each Agent defined as triple consisting of:
>> + SCMI agent_id,
>> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
>> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
>> +
>> +As an example:
>> +
>> +chosen {
>> + xen,scmi-secondary-agents = <
>> + 1 0x82000003 &scmi_shm_1
>> + 2 0x82000004 &scmi_shm_2
>> + 3 0x82000005 &scmi_shm_3
>> + 4 0x82000006 &scmi_shm_4>;
>> +}
>
> NIT: it should be };
>
> Looking at scmi_probe, collect_agents, and the following SCMI
> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
> information?
>
> It looks like we can discover the agend_ids for every channel, I guess
> what we need to know is the shmem location for every channel? But the
> full list of shmem channel is available below thanks to the scmi-shmem
> nodes.
>
> So, we have the list of scmi-shmem anyway, and we can probe the
> agent_id. The only parameter left is the smc_id/func_id.
>
> Or maybe smc_id/func_id can be calculated from agent_id?
>
> I am asking mostly because if a user is supposed to add this
> xen,scmi-secondary-agents property, where are they supposed to find the
> smc_id/func_id information?
>
> It is important that we write down in this document how the user is
> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
>
>
>> +/{
>> + scmi_shm_1: sram@47ff1000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>> + };
>> + scmi_shm_2: sram@47ff2000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>> + };
>> + scmi_shm_3: sram@47ff3000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>> + };
>> + scmi_shm_3: sram@47ff4000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>> + };
>
> Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
> Or are under /firmware? Or are they under /chosen?
>
> I take they are under the top level node together with scmi_shm_0?
>
> Can you please also clarify in the document as well?
>
>
>> +}
>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
>> index 8e50f6b7c7..bc3c64d6ec 100644
>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
>> SCMI nodes removed from Dom0/hwdom device tree.
>> (for example, thin Dom0 with Driver domain use-case).
>>
>> +### dom0_scmi_agent_id (ARM)
>> +> `= <integer>`
>> +
>> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
>> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
>> +The SCMI will be disabled for Dom0 if this option is not specified
>> +(for example, thin Dom0 or dom0less use-cases).
>> +The agent ids of domains existing on a single host must be unique.
>> +
>> ### dtuart (ARM)
>>> `= path [:options]`
>>
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index 28ba9eb787..7712f53cd4 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
>> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>> break;
>> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
>> + break;
>> default:
>> LOG(ERROR, "Unknown ARM_SCI type %d",
>> d_config->b_info.arch_arm.arm_sci.type);
>> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
>> index aa2190ab5b..11e31ce786 100644
>> --- a/tools/libs/light/libxl_types.idl
>> +++ b/tools/libs/light/libxl_types.idl
>> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
>>
>> libxl_arm_sci_type = Enumeration("arm_sci_type", [
>> (0, "none"),
>> - (1, "scmi_smc")
>> + (1, "scmi_smc"),
>> + (2, "scmi_smc_multiagent")
>> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
>>
>> libxl_arm_sci = Struct("arm_sci", [
>> ("type", libxl_arm_sci_type),
>> + ("agent_id", uint8)
>> ])
>>
>> libxl_rdm_reserve = Struct("rdm_reserve", [
>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>> index bd22be9d33..81aa3797e3 100644
>> --- a/tools/xl/xl_parse.c
>> +++ b/tools/xl/xl_parse.c
>> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
>> }
>> }
>>
>> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
>> + unsigned long val = parse_ulong(oparg);
>> +
>> + if (!val || val > 255) {
>> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
>> + val);
>> + ret = ERROR_INVAL;
>> + goto parse_error;
>> + }
>> + arm_sci->agent_id = val;
>> + }
>> +
>> ptr = strtok(NULL, ",");
>> }
>>
>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>> index 0a00f03a25..43d21eb889 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>> else if ( !strcmp(sci_type, "scmi_smc") )
>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
>> + {
>> + uint32_t agent_id = 0;
>> +
>> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
>> + !agent_id )
>
> shouldn't we check that agent_id <= 255 ?
>
>
>> + return -EINVAL;
>> +
>> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>> + d_cfg->arch.arm_sci_agent_id = agent_id;
>> + }
>> else
>> {
>> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 36d28b52a4..0c9274a2b3 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
>> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
>> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
>> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
>> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
>> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
>> continue;
>>
>> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") )
>> diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
>> index 5c5f0880c4..6b051c8ada 100644
>> --- a/xen/arch/arm/firmware/Kconfig
>> +++ b/xen/arch/arm/firmware/Kconfig
>> @@ -29,6 +29,17 @@ config SCMI_SMC
>> driver domain.
>> Use with EL3 firmware which supports only single SCMI OSPM agent.
>>
>> +config SCMI_SMC_MA
>> + bool "Enable ARM SCMI SMC multi-agent driver"
>> + select ARM_SCI
>> + help
>> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
>> + to EL3 firmware (TF-A) which supports multi-agent feature.
>> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
>> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
>> + allowed platform resources through dedicated SMC/HVC Shared memory based
>> + transport.
>> +
>> endchoice
>>
>> endmenu
>> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
>> index 71bdefc24a..37927e690e 100644
>> --- a/xen/arch/arm/firmware/Makefile
>> +++ b/xen/arch/arm/firmware/Makefile
>> @@ -1,2 +1,3 @@
>> obj-$(CONFIG_ARM_SCI) += sci.o
>> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
>> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
>> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
>> new file mode 100644
>> index 0000000000..3f4b9c5d6b
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-proto.h
>> @@ -0,0 +1,164 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Arm System Control and Management Interface definitions
>> + * Version 3.0 (DEN0056C)
>> + *
>> + * Copyright (c) 2024 EPAM Systems
>> + */
>> +
>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>
> NIT: ARM_FIRMWARE_SCMI_PROTO_H
>
>
>> +#include <xen/stdint.h>
>> +
>> +#define SCMI_SHORT_NAME_MAX_SIZE 16
>> +
>> +/* SCMI status codes. See section 4.1.4 */
>> +#define SCMI_SUCCESS 0
>> +#define SCMI_NOT_SUPPORTED (-1)
>> +#define SCMI_INVALID_PARAMETERS (-2)
>> +#define SCMI_DENIED (-3)
>> +#define SCMI_NOT_FOUND (-4)
>> +#define SCMI_OUT_OF_RANGE (-5)
>> +#define SCMI_BUSY (-6)
>> +#define SCMI_COMMS_ERROR (-7)
>> +#define SCMI_GENERIC_ERROR (-8)
>> +#define SCMI_HARDWARE_ERROR (-9)
>> +#define SCMI_PROTOCOL_ERROR (-10)
>> +
>> +/* Protocol IDs */
>> +#define SCMI_BASE_PROTOCOL 0x10
>> +
>> +/* Base protocol message IDs */
>> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
>> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
>> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
>> +#define SCMI_BASE_DISCOVER_AGENT 0x7
>> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
>> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
>> +
>> +typedef struct scmi_msg_header {
>> + uint8_t id;
>> + uint8_t type;
>> + uint8_t protocol;
>> + uint32_t status;
>> +} scmi_msg_header_t;
>> +
>> +/* Table 2 Message header format */
>> +#define SCMI_HDR_ID GENMASK(7, 0)
>> +#define SCMI_HDR_TYPE GENMASK(9, 8)
>> +#define SCMI_HDR_PROTO GENMASK(17, 10)
>> +
>> +#define SCMI_FIELD_GET(_mask, _reg) \
>> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
>> +#define SCMI_FIELD_PREP(_mask, _val) \
>> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
>> +
>> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
>> +{
>> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
>> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
>> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
>> +}
>> +
>> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
>> +{
>> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
>> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
>> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
>> +}
>> +
>> +static inline int scmi_to_xen_errno(int scmi_status)
>> +{
>> + if ( scmi_status == SCMI_SUCCESS )
>> + return 0;
>> +
>> + switch ( scmi_status )
>> + {
>> + case SCMI_NOT_SUPPORTED:
>> + return -EOPNOTSUPP;
>> + case SCMI_INVALID_PARAMETERS:
>> + return -EINVAL;
>> + case SCMI_DENIED:
>> + return -EACCES;
>> + case SCMI_NOT_FOUND:
>> + return -ENOENT;
>> + case SCMI_OUT_OF_RANGE:
>> + return -ERANGE;
>> + case SCMI_BUSY:
>> + return -EBUSY;
>> + case SCMI_COMMS_ERROR:
>> + return -ENOTCONN;
>> + case SCMI_GENERIC_ERROR:
>> + return -EIO;
>> + case SCMI_HARDWARE_ERROR:
>> + return -ENXIO;
>> + case SCMI_PROTOCOL_ERROR:
>> + return -EBADMSG;
>> + default:
>> + return -EINVAL;
>> + }
>> +}
>> +
>> +/* PROTOCOL_VERSION */
>> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
>> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
>> +
>> +struct scmi_msg_prot_version_p2a {
>> + uint32_t version;
>> +} __packed;
>> +
>> +/* BASE PROTOCOL_ATTRIBUTES */
>> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
>> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
>> +
>> +struct scmi_msg_base_attributes_p2a {
>> + uint32_t attributes;
>> +} __packed;
>> +
>> +/*
>> + * BASE_DISCOVER_AGENT
>> + */
>> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
>> +
>> +struct scmi_msg_base_discover_agent_a2p {
>> + uint32_t agent_id;
>> +} __packed;
>> +
>> +struct scmi_msg_base_discover_agent_p2a {
>> + uint32_t agent_id;
>> + char name[SCMI_SHORT_NAME_MAX_SIZE];
>> +} __packed;
>> +
>> +/*
>> + * BASE_SET_DEVICE_PERMISSIONS
>> + */
>> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
>> +
>> +struct scmi_msg_base_set_device_permissions_a2p {
>> + uint32_t agent_id;
>> + uint32_t device_id;
>> + uint32_t flags;
>> +} __packed;
>> +
>> +/*
>> + * BASE_RESET_AGENT_CONFIGURATION
>> + */
>> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
>> +
>> +struct scmi_msg_base_reset_agent_cfg_a2p {
>> + uint32_t agent_id;
>> + uint32_t flags;
>> +} __packed;
>> +
>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
>> new file mode 100644
>> index 0000000000..dd613ee0b5
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-shmem.c
>> @@ -0,0 +1,173 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>> + *
>> + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
>> + * Copyright (c) 2025 EPAM Systems
>> + */
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +
>> +#include <asm/io.h>
>> +#include <xen/err.h>
>> +
>> +#include "scmi-proto.h"
>> +#include "scmi-shmem.h"
>
> This code is written more generically than the description implies. If
> we only want to make SMC calls to TF-A on EL3 and exchange data with it
> over shared memory, then I think:
> - we don't need the __iomem tag, as there is no MMIO
agree
> - we only need a DMB, not a DSB (readl and writel imply DSB, use only
> readl_relaxed and writel_relaxed)
agree
>
> On the other hand, if we also want to handle the case where the SCMI
> server could be on a separate co-processor, then what this code is doing
> is not sufficient because we also need a dcache flush, in addition to
> the DSB.
>
> Bertrand, can you double-check?
If we want to handle a case where the memory is accessible to a coprocessor
but there is no cache coherency, we need to flush the dcache definitely.
Seeing the amount of data here, I do agree with Stefano that it would be a good
idea to make the provision to flush the data cache in all cases. Even if the data
is accessed by a secure partition or the firmware coherently, flushing in all cases
would have very limited performance impact here.
There is the other solution to have some kind of parameter to say if the accessor
has coherent cache access but I do not think the performance impact here would
justify such a complexity.
Cheers
Bertrand
>
>
>> +/*
>> + * Copy data from IO memory space to "real" memory space.
>> + */
>> +static void __memcpy_fromio(void *to, const volatile void __iomem *from,
>> + size_t count)
>> +{
>> + while ( count && !IS_ALIGNED((unsigned long)from, 4) )
>> + {
>> + *(u8 *)to = readb_relaxed(from);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +
>> + while ( count >= 4 )
>> + {
>> + *(u32 *)to = readl_relaxed(from);
>> + from += 4;
>> + to += 4;
>> + count -= 4;
>> + }
>> +
>> + while ( count )
>> + {
>> + *(u8 *)to = readb_relaxed(from);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +}
>> +
>> +/*
>> + * Copy data from "real" memory space to IO memory space.
>> + */
>> +static void __memcpy_toio(volatile void __iomem *to, const void *from,
>> + size_t count)
>> +{
>> + while ( count && !IS_ALIGNED((unsigned long)to, 4) )
>> + {
>> + writeb_relaxed(*(u8 *)from, to);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +
>> + while ( count >= 4 )
>> + {
>> + writel_relaxed(*(u32 *)from, to);
>> + from += 4;
>> + to += 4;
>> + count -= 4;
>> + }
>> +
>> + while ( count )
>> + {
>> + writeb_relaxed(*(u8 *)from, to);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +}
>
> I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
> we use a simple memcpy?
>
>
>> +static inline int
>> +shmem_channel_is_free(const volatile struct scmi_shared_mem __iomem *shmem)
>> +{
>> + return (readl(&shmem->channel_status) &
>> + SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE) ? 0 : -EBUSY;
>> +}
>> +
>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len)
>> +{
>> + int ret;
>> +
>> + if ( (len + sizeof(shmem->msg_header)) > SCMI_SHMEM_MAPPED_SIZE )
>> + {
>> + printk(XENLOG_ERR "scmi: Wrong size of smc message. Data is invalid\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = shmem_channel_is_free(shmem);
>> + if ( ret )
>> + return ret;
>> +
>> + writel_relaxed(0x0, &shmem->channel_status);
>> + /* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
>> + writel_relaxed(0x0, &shmem->flags);
>> + writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
>> + writel(pack_scmi_header(hdr), &shmem->msg_header);
>> +
>> + if ( len > 0 && data )
>> + __memcpy_toio(shmem->msg_payload, data, len);
>> +
>> + return 0;
>> +}
>> +
>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len)
>> +{
>> + int recv_len;
>> + int ret;
>> + int pad = sizeof(hdr->status);
>> +
>> + if ( len >= SCMI_SHMEM_MAPPED_SIZE - sizeof(shmem) )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Wrong size of input smc message. Data may be invalid\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = shmem_channel_is_free(shmem);
>> + if ( ret )
>> + return ret;
>> +
>> + recv_len = readl(&shmem->length) - sizeof(shmem->msg_header);
>> +
>> + if ( recv_len < 0 )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Wrong size of smc message. Data may be invalid\n");
>> + return -EINVAL;
>> + }
>> +
>> + unpack_scmi_header(readl(&shmem->msg_header), hdr);
>> +
>> + hdr->status = readl(&shmem->msg_payload);
>> + recv_len = recv_len > pad ? recv_len - pad : 0;
>> +
>> + ret = scmi_to_xen_errno(hdr->status);
>> + if ( ret )
>> + {
>> + printk(XENLOG_DEBUG "scmi: Error received: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + if ( recv_len > len )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Not enough buffer for message %d, expecting %d\n",
>> + recv_len, len);
>> + return -EINVAL;
>> + }
>> +
>> + if ( recv_len > 0 )
>> + __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
>> +
>> + return 0;
>> +}
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/arch/arm/firmware/scmi-shmem.h b/xen/arch/arm/firmware/scmi-shmem.h
>> new file mode 100644
>> index 0000000000..2f8e23ff76
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-shmem.h
>> @@ -0,0 +1,45 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Arm System Control and Management Interface definitions
>> + * Version 3.0 (DEN0056C)
>> + * Shared Memory based Transport
>> + *
>> + * Copyright (c) 2024 EPAM Systems
>> + */
>> +
>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
>> +#define XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
>
> NIT: ARM_FIRMWARE_SCMI_SHMEM_H
>
>
>> +#include <xen/stdint.h>
>> +
>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE BIT(0, UL)
>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_ERROR BIT(1, UL)
>> +
>> +struct scmi_shared_mem {
>> + uint32_t reserved;
>> + uint32_t channel_status;
>> + uint32_t reserved1[2];
>> + uint32_t flags;
>> + uint32_t length;
>> + uint32_t msg_header;
>> + uint8_t msg_payload[];
>> +};
>> +
>> +#define SCMI_SHMEM_MAPPED_SIZE PAGE_SIZE
>> +
>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len);
>> +
>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len);
>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_ */
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/arch/arm/firmware/scmi-smc-multiagent.c b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>> new file mode 100644
>> index 0000000000..e023bca3a1
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>> @@ -0,0 +1,860 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>> + *
>> + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
>> + * Copyright (c) 2025 EPAM Systems
>> + */
>> +
>> +#include <xen/acpi.h>
>> +
>> +#include <xen/device_tree.h>
>> +#include <xen/init.h>
>> +#include <xen/iocap.h>
>> +#include <xen/err.h>
>> +#include <xen/libfdt/libfdt.h>
>> +#include <xen/param.h>
>> +#include <xen/sched.h>
>> +#include <xen/vmap.h>
>> +
>> +#include <asm/firmware/sci.h>
>> +#include <asm/smccc.h>
>> +
>> +#include "scmi-proto.h"
>> +#include "scmi-shmem.h"
>> +
>> +#define SCMI_AGENT_ID_INVALID 0xFF
>> +
>> +static uint8_t __initdata opt_dom0_scmi_agent_id = SCMI_AGENT_ID_INVALID;
>> +integer_param("dom0_scmi_agent_id", opt_dom0_scmi_agent_id);
>> +
>> +#define SCMI_SECONDARY_AGENTS "xen,scmi-secondary-agents"
>> +
>> +#define HYP_CHANNEL 0x0
>> +
>> +struct scmi_channel {
>> + uint32_t agent_id;
>> + uint32_t func_id;
>> + domid_t domain_id;
>> + uint64_t paddr;
>> + uint64_t len;
>> + struct scmi_shared_mem __iomem *shmem;
>> + spinlock_t lock;
>> + struct list_head list;
>> +};
>> +
>> +struct scmi_data {
>> + struct list_head channel_list;
>> + spinlock_t channel_list_lock;
>> + uint32_t func_id;
>> + bool initialized;
>> + uint32_t shmem_phandle;
>> + struct dt_device_node *dt_dev;
>> +};
>> +
>> +static struct scmi_data scmi_data;
>> +
>> +static int send_smc_message(struct scmi_channel *chan_info,
>> + scmi_msg_header_t *hdr, void *data, int len)
>> +{
>> + struct arm_smccc_res resp;
>> + int ret;
>> +
>> + ret = shmem_put_message(chan_info->shmem, hdr, data, len);
>> + if ( ret )
>> + return ret;
>> +
>> + arm_smccc_1_1_smc(chan_info->func_id, 0, 0, 0, 0, 0, 0, 0, &resp);
>> +
>> + if ( resp.a0 )
>> + return -EOPNOTSUPP;
>
> Why if repo.a0 != 0 then we assume -EOPNOTSUPP? Is this part of the SCMI
> specification?
>
>
>> + return 0;
>> +}
>> +
>> +static int do_smc_xfer(struct scmi_channel *chan_info, scmi_msg_header_t *hdr,
>> + void *tx_data, int tx_size, void *rx_data, int rx_size)
>> +{
>> + int ret = 0;
>> +
>> + ASSERT(chan_info && chan_info->shmem);
>> +
>> + if ( !hdr )
>> + return -EINVAL;
>> +
>> + spin_lock(&chan_info->lock);
>> +
>> + printk(XENLOG_DEBUG
>> + "scmi: agent_id = %d msg_id = %x type = %d, proto = %x\n",
>> + chan_info->agent_id, hdr->id, hdr->type, hdr->protocol);
>> +
>> + ret = send_smc_message(chan_info, hdr, tx_data, tx_size);
>> + if ( ret )
>> + goto clean;
>> +
>> + ret = shmem_get_response(chan_info->shmem, hdr, rx_data, rx_size);
>> +
>> +clean:
>> + printk(XENLOG_DEBUG
>> + "scmi: get smc response agent_id = %d msg_id = %x proto = %x res=%d\n",
>> + chan_info->agent_id, hdr->id, hdr->protocol, ret);
>> +
>> + spin_unlock(&chan_info->lock);
>> +
>> + return ret;
>> +}
>> +
>> +static struct scmi_channel *get_channel_by_id(uint32_t agent_id)
>> +{
>> + struct scmi_channel *curr;
>> + bool found = false;
>> +
>> + spin_lock(&scmi_data.channel_list_lock);
>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>> + {
>> + if ( curr->agent_id == agent_id )
>> + {
>> + found = true;
>> + break;
>> + }
>> + }
>> +
>> + spin_unlock(&scmi_data.channel_list_lock);
>> + if ( found )
>> + return curr;
>> +
>> + return NULL;
>> +}
>> +
>> +static struct scmi_channel *acquire_scmi_channel(struct domain *d,
>> + uint32_t agent_id)
>> +{
>> + struct scmi_channel *curr;
>> + struct scmi_channel *ret = ERR_PTR(-ENOENT);
>> +
>> + spin_lock(&scmi_data.channel_list_lock);
>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>> + {
>> + if ( curr->agent_id == agent_id )
>> + {
>> + if ( curr->domain_id != DOMID_INVALID )
>> + {
>> + ret = ERR_PTR(-EEXIST);
>> + break;
>> + }
>> +
>> + curr->domain_id = d->domain_id;
>> + ret = curr;
>> + break;
>> + }
>> + }
>> +
>> + spin_unlock(&scmi_data.channel_list_lock);
>> +
>> + return ret;
>> +}
>> +
>> +static void relinquish_scmi_channel(struct scmi_channel *channel)
>> +{
>> + ASSERT(channel != NULL);
>> +
>> + spin_lock(&scmi_data.channel_list_lock);
>> + channel->domain_id = DOMID_INVALID;
>> + spin_unlock(&scmi_data.channel_list_lock);
>> +}
>> +
>> +static int map_channel_memory(struct scmi_channel *channel)
>> +{
>> + ASSERT(channel && channel->paddr);
>> + channel->shmem = ioremap_nocache(channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
>
> ioremap is for MMIO, if these shared memory channels are on DDR, then it
> would not be the right call. Are the "arm,scmi-shmem" address ranges
> part of the memory node ranges? Or are they completely separate?
>
> Also, why nocache? Wouldn't we want ioremap_cache?
>
>
>> + if ( !channel->shmem )
>> + return -ENOMEM;
>> +
>> + channel->shmem->channel_status = SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE;
>> + printk(XENLOG_DEBUG "scmi: Got shmem %lx after vmap %p\n", channel->paddr,
>> + channel->shmem);
>> +
>> + return 0;
>> +}
>> +
>> +static void unmap_channel_memory(struct scmi_channel *channel)
>> +{
>> + ASSERT(channel && channel->shmem);
>> + iounmap(channel->shmem);
>> + channel->shmem = NULL;
>> +}
>> +
>> +static struct scmi_channel *smc_create_channel(uint32_t agent_id,
>> + uint32_t func_id, uint64_t addr)
>> +{
>> + struct scmi_channel *channel;
>> +
>> + channel = get_channel_by_id(agent_id);
>> + if ( channel )
>> + return ERR_PTR(EEXIST);
>> +
>> + channel = xmalloc(struct scmi_channel);
>> + if ( !channel )
>> + return ERR_PTR(ENOMEM);
>> +
>> + spin_lock_init(&channel->lock);
>> + channel->agent_id = agent_id;
>> + channel->func_id = func_id;
>> + channel->domain_id = DOMID_INVALID;
>> + channel->shmem = NULL;
>> + channel->paddr = addr;
>> + list_add_tail(&channel->list, &scmi_data.channel_list);
>> + return channel;
>> +}
>> +
>> +static void free_channel_list(void)
>> +{
>> + struct scmi_channel *curr, *_curr;
>> +
>> + list_for_each_entry_safe(curr, _curr, &scmi_data.channel_list, list)
>> + {
>> + list_del(&curr->list);
>> + xfree(curr);
>> + }
>> +}
>> +
>> +static int __init
>> +scmi_dt_read_hyp_channel_addr(struct dt_device_node *scmi_node, u64 *addr,
>> + u64 *size)
>> +{
>> + struct dt_device_node *shmem_node;
>> + const __be32 *prop;
>> +
>> + prop = dt_get_property(scmi_node, "shmem", NULL);
>> + if ( !prop )
>> + return -EINVAL;
>> +
>> + shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
>> + if ( IS_ERR_OR_NULL(shmem_node) )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Device tree error, can't parse reserved memory %ld\n",
>> + PTR_ERR(shmem_node));
>> + return PTR_ERR(shmem_node);
>> + }
>> +
>> + return dt_device_get_address(shmem_node, 0, addr, size);
>> +}
>> +
>> +/*
>> + * Handle Dom0 SCMI specific DT nodes
>> + *
>> + * Make a decision on copying SCMI specific nodes into Dom0 device tree.
>> + * For SCMI multi-agent case:
>> + * - shmem nodes will not be copied and generated instead if SCMI
>> + * is enabled for Dom0
>> + * - scmi node will be copied if SCMI is enabled for Dom0
>> + */
>> +static bool scmi_dt_handle_node(struct domain *d, struct dt_device_node *node)
>> +{
>> + static const struct dt_device_match skip_matches[] __initconst = {
>> + DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
>> + { /* sentinel */ },
>> + };
>> + static const struct dt_device_match scmi_matches[] __initconst = {
>> + DT_MATCH_PATH("/firmware/scmi"),
>> + { /* sentinel */ },
>> + };
>> +
>> + if ( !scmi_data.initialized )
>> + return false;
>> +
>> + /* always drop shmem */
>> + if ( dt_match_node(skip_matches, node) )
>> + {
>> + dt_dprintk(" Skip scmi shmem\n");
>> + return true;
>> + }
>> +
>> + /* drop scmi if not enabled */
>> + if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
>> + {
>> + dt_dprintk(" Skip scmi node\n");
>> + return true;
>> + }
>> +
>> + return false;
>> +}
>> +
>> +/*
>> + * Finalize Dom0 SCMI specific DT nodes
>> + *
>> + * if SCMI is enabled for Dom0:
>> + * - generate shmem node
>> + * - map SCMI shmem MMIO into Dom0
>> + */
>> +static int scmi_dt_finalize(struct domain *d, void *fdt)
>> +{
>> + __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
>> + struct scmi_channel *channel;
>> + int nodeoffset;
>> + __be32 *cells;
>> + __be32 val;
>> + char buf[64];
>> + int res, rc;
>> +
>> + if ( !sci_domain_is_enabled(d) )
>> + return 0;
>> +
>> + channel = d->arch.sci_data;
>> +
>> + /*
>> + * Replace "arm,smc-id" with proper value assigned for Dom0 SCMI channel
>> + */
>> + nodeoffset = fdt_node_offset_by_compatible(fdt, -1, "arm,scmi-smc");
>> + if ( nodeoffset < 0 )
>> + return -ENODEV;
>> +
>> + cells = (__be32 *)&val;
>> + dt_set_cell(&cells, 1, channel->func_id);
>> + res = fdt_setprop_inplace(fdt, nodeoffset, "arm,smc-id", &val, sizeof(val));
>> + if ( res )
>> + return -EINVAL;
>> +
>
> Are you sure it is worth to go through all this trouble to modify FDT in
> place when we could simply generate the DT node from scratch like we do
> for example for the GIC? This seems to be more error prone as well. Is
> generating it from scratch is really difficult? If it is difficult then OK.
>
>
>> + /*
>> + * All SCMI shmem nodes should be removed from Dom0 DT at this point, so
>> + * the shmem node for Dom0 need to be generated from SCMI channel assigned
>> + * to Dom0.
>> + * The original SCMI shmem node from platform DT is used by Xen SCMI driver
>> + * itself as privileged channel (agent_id=0) to manage other SCMI
>> + * agents (domains).
>> + */
>> + snprintf(buf, sizeof(buf), "scmi-shmem@%lx", channel->paddr);
>> +
>> + res = fdt_begin_node(fdt, buf);
>> + if ( res )
>> + return res;
>> +
>> + res = fdt_property_string(fdt, "compatible", "arm,scmi-shmem");
>> + if ( res )
>> + return res;
>> +
>> + cells = ®[0];
>> +
>> + dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>> + channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
>> +
>> + res = fdt_property(fdt, "reg", reg, sizeof(reg));
>> + if ( res )
>> + return res;
>> +
>> + res = fdt_property_cell(fdt, "phandle", scmi_data.shmem_phandle);
>> + if ( res )
>> + return res;
>> +
>> + res = fdt_end_node(fdt);
>> + if ( res )
>> + return res;
>> +
>> + /*
>> + * Map SCMI shmem into Dom0 here as shmem nodes are excluded from
>> + * generic Dom0 DT processing
>> + */
>> + res = iomem_permit_access(d, paddr_to_pfn(channel->paddr),
>> + paddr_to_pfn(channel->paddr +
>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>> + if ( res )
>> + return res;
>> +
>> + res = map_regions_p2mt(d, gaddr_to_gfn(channel->paddr),
>> + PFN_UP(SCMI_SHMEM_MAPPED_SIZE),
>> + maddr_to_mfn(channel->paddr), p2m_mmio_direct_nc);
>> + if ( res )
>> + {
>> + rc = iomem_deny_access(d, paddr_to_pfn(channel->paddr),
>> + paddr_to_pfn(channel->paddr +
>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>> + if ( rc )
>> + printk(XENLOG_ERR "scmi: Unable to deny iomem access , err = %d\n",
>> + rc);
>> + }
>> +
>> + return res;
>> +}
>> +
>> +static int scmi_assign_device(uint32_t agent_id, uint32_t device_id,
>> + uint32_t flags)
>> +{
>> + struct scmi_msg_base_set_device_permissions_a2p tx;
>> + struct scmi_channel *channel;
>> + scmi_msg_header_t hdr;
>> + int ret;
>> +
>> + channel = get_channel_by_id(HYP_CHANNEL);
>> + if ( !channel )
>> + return -EINVAL;
>> +
>> + hdr.id = SCMI_BASE_SET_DEVICE_PERMISSIONS;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + tx.agent_id = agent_id;
>> + tx.device_id = device_id;
>> + tx.flags = flags;
>> +
>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>> + if ( ret == -EOPNOTSUPP )
>> + return 0;
>
> Is it actually OK to pretend that everything worked if the return is
> -EOPNOTSUPP? I mean that in this case can we assume that the device is
> actually assigned anyway? Wouldn't follow up SCMI operations on this
> device fail?
>
>
>> + return ret;
>> +}
>> +
>> +static int scmi_dt_assign_device(struct domain *d,
>> + struct dt_phandle_args *ac_spec)
>> +{
>> + struct scmi_channel *agent_channel;
>> + uint32_t scmi_device_id = ac_spec->args[0];
>> + int ret;
>> +
>> + if ( !d->arch.sci_data )
>> + return 0;
>> +
>> + /* The access-controllers is specified for DT dev, but it's not a SCMI */
>> + if ( ac_spec->np != scmi_data.dt_dev )
>> + return 0;
>
> I wonder if this should be an error
>
>
>> + agent_channel = d->arch.sci_data;
>> +
>> + spin_lock(&agent_channel->lock);
>> +
>> + ret = scmi_assign_device(agent_channel->agent_id, scmi_device_id,
>> + SCMI_BASE_DEVICE_ACCESS_ALLOW);
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: could not assign dev for %pd agent:%d dev_id:%u (%d)",
>> + d, agent_channel->agent_id, scmi_device_id, ret);
>> + }
>> +
>> + spin_unlock(&agent_channel->lock);
>> + return ret;
>> +}
>> +
>> +static __init int collect_agents(struct dt_device_node *scmi_node)
>> +{
>> + const struct dt_device_node *chosen_node;
>> + const __be32 *prop;
>> + uint32_t len, i;
>> +
>> + chosen_node = dt_find_node_by_path("/chosen");
>> + if ( !chosen_node )
>> + {
>> + printk(XENLOG_ERR "scmi: chosen node not found\n");
>> + return -ENOENT;
>> + }
>> +
>> + prop = dt_get_property(chosen_node, SCMI_SECONDARY_AGENTS, &len);
>> + if ( !prop )
>> + {
>> + printk(XENLOG_WARNING "scmi: No %s property found\n",
>> + SCMI_SECONDARY_AGENTS);
>> + return -ENODEV;
>> + }
>> +
>> + if ( len % (3 * sizeof(uint32_t)) )
>> + {
>> + printk(XENLOG_ERR "scmi: Invalid length of %s property: %d\n",
>> + SCMI_SECONDARY_AGENTS, len);
>> + return -EINVAL;
>> + }
>> +
>> + for ( i = 0; i < len / (3 * sizeof(uint32_t)); i++ )
>> + {
>> + uint32_t agent_id = be32_to_cpu(*prop++);
>> + uint32_t smc_id = be32_to_cpu(*prop++);
>> + uint32_t shmem_phandle = be32_to_cpu(*prop++);
>> + struct dt_device_node *node = dt_find_node_by_phandle(shmem_phandle);
>> + u64 addr, size;
>> + int ret;
>> +
>> + if ( !node )
>> + {
>> + printk(XENLOG_ERR "scmi: Could not find shmem node for agent %d\n",
>> + agent_id);
>> + return -EINVAL;
>> + }
>> +
>> + ret = dt_device_get_address(node, 0, &addr, &size);
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Could not read shmem address for agent %d: %d",
>> + agent_id, ret);
>> + return ret;
>> + }
>> +
>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>> + {
>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = PTR_RET(smc_create_channel(agent_id, smc_id, addr));
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR "scmi: Could not create channel for agent %d: %d",
>> + agent_id, ret);
>> + return ret;
>> + }
>> +
>> + printk(XENLOG_DEBUG "scmi: Agent %d SMC %X addr %lx\n", agent_id,
>> + smc_id, addr);
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int scmi_domain_init(struct domain *d,
>> + struct xen_domctl_createdomain *config)
>> +{
>> + struct scmi_channel *channel;
>> + int ret;
>> +
>> + if ( !scmi_data.initialized )
>> + return 0;
>> +
>> + /*
>> + * Special case for Dom0 - the SCMI support is enabled basing on
>> + * "dom0_sci_agent_id" Xen command line parameter
>> + */
>> + if ( is_hardware_domain(d) )
>> + {
>> + if ( opt_dom0_scmi_agent_id != SCMI_AGENT_ID_INVALID )
>> + {
>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>> + config->arch.arm_sci_agent_id = opt_dom0_scmi_agent_id;
>> + }
>> + else
>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>> + }
>> +
>> + if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE )
>> + return 0;
>> +
>> + channel = acquire_scmi_channel(d, config->arch.arm_sci_agent_id);
>> + if ( IS_ERR(channel) )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Failed to acquire SCMI channel for agent_id %u: %ld\n",
>> + config->arch.arm_sci_agent_id, PTR_ERR(channel));
>> + return PTR_ERR(channel);
>> + }
>> +
>> + printk(XENLOG_INFO
>> + "scmi: Acquire channel id = 0x%x, domain_id = %d paddr = 0x%lx\n",
>> + channel->agent_id, channel->domain_id, channel->paddr);
>> +
>> + /*
>> + * Dom0 (if present) needs to have an access to the guest memory range
>> + * to satisfy iomem_access_permitted() check in XEN_DOMCTL_iomem_permission
>> + * domctl.
>
> Ideally this should not be needed but I understand we don't have an
> easy solution, I think we can go ahead with this for now.
>
>
>> + */
>> + if ( hardware_domain && !is_hardware_domain(d) )
>> + {
>> + ret = iomem_permit_access(hardware_domain, paddr_to_pfn(channel->paddr),
>> + paddr_to_pfn(channel->paddr + PAGE_SIZE - 1));
>> + if ( ret )
>> + goto error;
>> + }
>> +
>> + d->arch.sci_data = channel;
>> + d->arch.sci_enabled = true;
>> +
>> + return 0;
>> +
>> +error:
>> + relinquish_scmi_channel(channel);
>> + return ret;
>> +}
>> +
>> +int scmi_domain_sanitise_config(struct xen_domctl_createdomain *config)
>> +{
>> + if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
>> + config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA )
>> + {
>> + dprintk(XENLOG_INFO, "scmi: Unsupported ARM_SCI type\n");
>> + return -EINVAL;
>> + }
>> + else if ( config->arch.arm_sci_type ==
>> + XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA &&
>> + config->arch.arm_sci_agent_id == 0 )
>> + {
>> + dprintk(XENLOG_INFO,
>> + "scmi: A zero ARM SCMI agent_id is not supported\n");
>> + return -EINVAL;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int scmi_relinquish_resources(struct domain *d)
>> +{
>> + int ret;
>> + struct scmi_channel *channel, *agent_channel;
>> + scmi_msg_header_t hdr;
>> + struct scmi_msg_base_reset_agent_cfg_a2p tx;
>> +
>> + if ( !d->arch.sci_data )
>> + return 0;
>> +
>> + agent_channel = d->arch.sci_data;
>> +
>> + spin_lock(&agent_channel->lock);
>> + tx.agent_id = agent_channel->agent_id;
>> + spin_unlock(&agent_channel->lock);
>> +
>> + channel = get_channel_by_id(HYP_CHANNEL);
>> + if ( !channel )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Unable to get Hypervisor scmi channel for domain %d\n",
>> + d->domain_id);
>> + return -EINVAL;
>> + }
>> +
>> + hdr.id = SCMI_BASE_RESET_AGENT_CONFIGURATION;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + tx.flags = 0;
>> +
>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>> + if ( ret == -EOPNOTSUPP )
>> + return 0;
>> +
>> + return ret;
>> +}
>> +
>> +static void scmi_domain_destroy(struct domain *d)
>> +{
>> + struct scmi_channel *channel;
>> +
>> + if ( !d->arch.sci_data )
>> + return;
>> +
>> + channel = d->arch.sci_data;
>> + spin_lock(&channel->lock);
>> +
>> + relinquish_scmi_channel(channel);
>> + printk(XENLOG_DEBUG "scmi: Free domain %d\n", d->domain_id);
>> +
>> + d->arch.sci_data = NULL;
>> + d->arch.sci_enabled = true;
>> +
>> + spin_unlock(&channel->lock);
>> +}
>> +
>> +static bool scmi_handle_call(struct cpu_user_regs *regs)
>> +{
>> + uint32_t fid = (uint32_t)get_user_reg(regs, 0);
>> + struct scmi_channel *agent_channel;
>> + struct domain *d = current->domain;
>> + struct arm_smccc_res resp;
>> + bool res = false;
>> +
>> + if ( !sci_domain_is_enabled(d) )
>> + return false;
>> +
>> + agent_channel = d->arch.sci_data;
>> + spin_lock(&agent_channel->lock);
>> +
>> + if ( agent_channel->func_id != fid )
>> + {
>> + res = false;
>> + goto unlock;
>> + }
>> +
>> + arm_smccc_1_1_smc(fid,
>> + get_user_reg(regs, 1),
>> + get_user_reg(regs, 2),
>> + get_user_reg(regs, 3),
>> + get_user_reg(regs, 4),
>> + get_user_reg(regs, 5),
>> + get_user_reg(regs, 6),
>> + get_user_reg(regs, 7),
>> + &resp);
>> +
>> + set_user_reg(regs, 0, resp.a0);
>> + set_user_reg(regs, 1, resp.a1);
>> + set_user_reg(regs, 2, resp.a2);
>> + set_user_reg(regs, 3, resp.a3);
>> + res = true;
>> +unlock:
>> + spin_unlock(&agent_channel->lock);
>> +
>> + return res;
>> +}
>> +
>> +static const struct sci_mediator_ops scmi_ops = {
>> + .domain_init = scmi_domain_init,
>> + .domain_destroy = scmi_domain_destroy,
>> + .relinquish_resources = scmi_relinquish_resources,
>> + .handle_call = scmi_handle_call,
>> + .dom0_dt_handle_node = scmi_dt_handle_node,
>> + .dom0_dt_finalize = scmi_dt_finalize,
>> + .domain_sanitise_config = scmi_domain_sanitise_config,
>> + .assign_dt_device = scmi_dt_assign_device,
>> +};
>> +
>> +static int __init scmi_check_smccc_ver(void)
>> +{
>> + if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
>> + {
>> + printk(XENLOG_WARNING
>> + "scmi: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
>> + return -ENOSYS;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static __init int scmi_probe(struct dt_device_node *scmi_node, const void *data)
>> +{
>> + u64 addr, size;
>> + int ret, i;
>> + struct scmi_channel *channel, *agent_channel;
>> + int n_agents;
>> + scmi_msg_header_t hdr;
>> + struct scmi_msg_base_attributes_p2a rx;
>> +
>> + ASSERT(scmi_node != NULL);
>> +
>> + INIT_LIST_HEAD(&scmi_data.channel_list);
>> + spin_lock_init(&scmi_data.channel_list_lock);
>> +
>> + if ( !acpi_disabled )
>> + {
>> + printk(XENLOG_WARNING "scmi: is not supported when using ACPI\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = scmi_check_smccc_ver();
>> + if ( ret )
>> + return ret;
>> +
>> + if ( !dt_property_read_u32(scmi_node, "arm,smc-id", &scmi_data.func_id) )
>> + {
>> + printk(XENLOG_ERR "scmi: unable to read smc-id from DT\n");
>> + return -ENOENT;
>> + }
>> +
>> + /* save shmem phandle and re-use it fro Dom0 DT shmem node */
>> + if ( !dt_property_read_u32(scmi_node, "shmem", &scmi_data.shmem_phandle) )
>> + {
>> + printk(XENLOG_ERR "scmi: unable to read shmem phandle from DT\n");
>> + return -ENOENT;
>> + }
>> +
>> + ret = scmi_dt_read_hyp_channel_addr(scmi_node, &addr, &size);
>> + if ( IS_ERR_VALUE(ret) )
>> + return -ENOENT;
>> +
>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>> + {
>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>> + return -EINVAL;
>> + }
>> +
>> + scmi_data.dt_dev = scmi_node;
>> +
>> + channel = smc_create_channel(HYP_CHANNEL, scmi_data.func_id, addr);
>> + if ( IS_ERR(channel) )
>> + goto out;
>> +
>> + ret = map_channel_memory(channel);
>> + if ( ret )
>> + goto out;
>> +
>> + channel->domain_id = DOMID_XEN;
>> +
>> + hdr.id = SCMI_BASE_PROTOCOL_ATTIBUTES;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + ret = do_smc_xfer(channel, &hdr, NULL, 0, &rx, sizeof(rx));
>> + if ( ret )
>> + goto error;
>> +
>> + n_agents = SCMI_FIELD_GET(SCMI_BASE_ATTR_NUM_AGENT, rx.attributes);
>> + printk(XENLOG_DEBUG "scmi: Got agent count %d\n", n_agents);
>> +
>> + ret = collect_agents(scmi_node);
>> + if ( ret )
>> + goto error;
>> +
>> + i = 1;
>> +
>> + list_for_each_entry(agent_channel, &scmi_data.channel_list, list)
>> + {
>> + struct scmi_msg_base_discover_agent_p2a da_rx;
>> + struct scmi_msg_base_discover_agent_a2p da_tx;
>> +
>> + ret = map_channel_memory(agent_channel);
>> + if ( ret )
>> + goto error;
>> +
>> + hdr.id = SCMI_BASE_DISCOVER_AGENT;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + da_tx.agent_id = agent_channel->agent_id;
>> +
>> + ret = do_smc_xfer(agent_channel, &hdr, &da_tx, sizeof(da_tx), &da_rx,
>> + sizeof(da_rx));
>> + if ( agent_channel->domain_id != DOMID_XEN )
>> + unmap_channel_memory(agent_channel);
>> + if ( ret )
>> + goto error;
>> +
>> + printk(XENLOG_DEBUG "id=0x%x name=%s\n", da_rx.agent_id, da_rx.name);
>> +
>> + agent_channel->agent_id = da_rx.agent_id;
>
> It is OK to set agent_channel->agent_id to the value provided by the
> SCMI server, but if we are also taking the agent_channel->agent_id value
> from the user via device tree, shouldn't we throw an error if there is a
> mismatch?
>
> Or even better: can we avoid taking the value via device tree to make it
> easier to configure?
>
>
>> + if ( i > n_agents )
>> + break;
>> +
>> + i++;
>> + }
>> +
>> + ret = sci_register(&scmi_ops);
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR "SCMI: mediator already registered (ret = %d)\n",
>> + ret);
>> + return ret;
>> + }
>> +
>> + scmi_data.initialized = true;
>> + goto out;
>> +
>> +error:
>> + unmap_channel_memory(channel);
>> + free_channel_list();
>> +out:
>> + return ret;
>> +}
>> +
>> +static const struct dt_device_match scmi_smc_match[] __initconst = {
>> + DT_MATCH_COMPATIBLE("arm,scmi-smc"),
>> + { /* sentinel */ },
>> +};
>> +
>> +DT_DEVICE_START(scmi_smc_ma, "SCMI SMC MEDIATOR", DEVICE_FIRMWARE)
>> + .dt_match = scmi_smc_match,
>> + .init = scmi_probe,
>> +DT_DEVICE_END
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
>> index 095b1a23e3..30e46de6d7 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -329,6 +329,7 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>>
>> #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
>> #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC 1
>> +#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA 2
>>
>> struct xen_arch_domainconfig {
>> /* IN/OUT */
>> @@ -355,6 +356,8 @@ struct xen_arch_domainconfig {
>> uint32_t clock_frequency;
>> /* IN */
>> uint8_t arm_sci_type;
>> + /* IN */
>> + uint8_t arm_sci_agent_id;
>> };
>> #endif /* __XEN__ || __XEN_TOOLS__ */
>>
>> --
>> 2.34.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
` (7 preceding siblings ...)
2025-05-19 15:50 ` [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent Oleksii Moisieiev
@ 2025-06-05 22:40 ` Julien Grall
8 siblings, 0 replies; 67+ messages in thread
From: Julien Grall @ 2025-06-05 22:40 UTC (permalink / raw)
To: Oleksii Moisieiev, xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Michal Orzel, Roger Pau Monné,
Stefano Stabellini, Volodymyr Babchuk, Grygorii Strashko
Hi Oleksii,
On 19/05/2025 16:50, Oleksii Moisieiev wrote:
> > MAINTAINERS | 6 +
> .../arm/firmware/arm-scmi-proposal.rst | 224 +++++
> .../arm/firmware/arm-scmi.rst | 442 +++++++++
> docs/hypervisor-guide/arm/index.rst | 9 +
> docs/hypervisor-guide/index.rst | 1 +
> docs/man/xl.cfg.5.pod.in | 47 +
> docs/misc/arm/device-tree/booting.txt | 75 ++
> docs/misc/xen-command-line.pandoc | 18 +
> tools/include/libxl.h | 5 +
> tools/libs/light/libxl_arm.c | 18 +
> tools/libs/light/libxl_types.idl | 12 +
> tools/xl/xl_parse.c | 48 +
> xen/arch/arm/device.c | 5 +
> xen/arch/arm/dom0less-build.c | 49 +
> xen/arch/arm/domain.c | 12 +-
> xen/arch/arm/domain_build.c | 11 +-
> xen/arch/arm/firmware/Kconfig | 36 +-
> xen/arch/arm/firmware/Makefile | 2 +
> xen/arch/arm/firmware/sci.c | 191 ++++
> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
> xen/arch/arm/firmware/scmi-shmem.h | 45 +
> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++
> xen/arch/arm/firmware/scmi-smc.c | 191 +++-
> xen/arch/arm/include/asm/domain.h | 5 +
> xen/arch/arm/include/asm/firmware/sci.h | 214 +++++
> xen/arch/arm/include/asm/firmware/scmi-smc.h | 41 -
> xen/arch/arm/vsmc.c | 4 +-
> xen/common/domctl.c | 19 +
> xen/drivers/passthrough/device_tree.c | 6 +
> xen/include/asm-generic/device.h | 1 +
> xen/include/public/arch-arm.h | 8 +
> 32 files changed, 2856 insertions(+), 86 deletions(-)
> create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi.rst
> create mode 100644 docs/hypervisor-guide/arm/index.rst
> create mode 100644 xen/arch/arm/firmware/sci.c
> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
> create mode 100644 xen/arch/arm/include/asm/firmware/sci.h
> delete mode 100644 xen/arch/arm/include/asm/firmware/scmi-smc.h
Looking at the list above, I don't see an update to SUPPORT.md. So I
believe, this would fall under:
### Arm: SCMI over SMC calls forwarding to EL3 Firmware
Enable SCMI calls using SMC as doorbell mechanism and Shared Memory for
transport ("arm,scmi-smc" compatible only) to reach EL3 Firmware if issued
by hwdom. Some platforms use SCMI for access to system-level resources.
Status: Supported
I think this should be updated for the multi-agent support to tech preview.
Cheers,
>
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-05-23 20:06 ` Stefano Stabellini
2025-06-02 7:17 ` Bertrand Marquis
@ 2025-06-05 22:44 ` Julien Grall
2025-06-12 12:03 ` Oleksii Moisieiev
2 siblings, 0 replies; 67+ messages in thread
From: Julien Grall @ 2025-06-05 22:44 UTC (permalink / raw)
To: Stefano Stabellini, Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
Hi Oleksii & Stefano,
On 23/05/2025 21:06, Stefano Stabellini wrote:
>> +SCMI SMC multi-agent support
>> +============================
>> +
>> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
>> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
>> +according to the SCMI compliant EL3 Firmware specification with
>> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
>> +the top-level "chosen" node:
>> +
>> +- xen,scmi-secondary-agents
>> +
>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
>> + available for Xen. Each Agent defined as triple consisting of:
>> + SCMI agent_id,
>> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
>> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
>> +
>> +As an example:
>> +
>> +chosen {
>> + xen,scmi-secondary-agents = <
>> + 1 0x82000003 &scmi_shm_1
>> + 2 0x82000004 &scmi_shm_2
>> + 3 0x82000005 &scmi_shm_3
>> + 4 0x82000006 &scmi_shm_4>;
>> +}
>
> NIT: it should be };
>
> Looking at scmi_probe, collect_agents, and the following SCMI
> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
> information?
>
> It looks like we can discover the agend_ids for every channel, I guess
> what we need to know is the shmem location for every channel? But the
> full list of shmem channel is available below thanks to the scmi-shmem
> nodes.
>
> So, we have the list of scmi-shmem anyway, and we can probe the
> agent_id. The only parameter left is the smc_id/func_id.
>
> Or maybe smc_id/func_id can be calculated from agent_id?
>
> I am asking mostly because if a user is supposed to add this
> xen,scmi-secondary-agents property, where are they supposed to find the
> smc_id/func_id information?
>
> It is important that we write down in this document how the user is
> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
I was actually going to write something similar to what you wrote. I
think this is error-prone to let the user specify the full SMC ID. It
would be better if this can be computed from a base + an agent ID.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-05-22 0:25 ` Stefano Stabellini
2025-05-22 6:18 ` Jan Beulich
@ 2025-06-12 11:42 ` Oleksii Moisieiev
2025-06-12 13:10 ` Grygorii Strashko
2025-06-18 0:04 ` Stefano Stabellini
1 sibling, 2 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-12 11:42 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
Hi Stefano,
I'm very sorry for a long silence. Please see my answers below:
On 22/05/2025 03:25, Stefano Stabellini wrote:
> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>
>> Add chained handling of assigned DT devices to support access-controller
>> functionality through SCI framework, so DT device assign request can be
>> passed to FW for processing and enabling VM access to requested device
>> (for example, device power management through FW interface like SCMI).
>>
>> The SCI access-controller DT device processing is chained after IOMMU
>> processing and expected to be executed for any DT device regardless of its
>> protection by IOMMU (or if IOMMU is disabled).
>>
>> This allows to pass not only IOMMU protected DT device through
>> xl.cfg:"dtdev" property for processing:
>>
>> dtdev = [
>> "/soc/video@e6ef0000", <- IOMMU protected device
>> "/soc/i2c@e6508000", <- not IOMMU protected device
>> ]
>>
>> The change is done in two parts:
>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>> not fail if DT device is not protected by IOMMU
>> 2) add chained call to sci_do_domctl() in do_domctl()
>>
>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>> ---
>>
>>
>>
>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>> xen/common/domctl.c | 19 +++++++++++++
>> xen/drivers/passthrough/device_tree.c | 6 ++++
>> 4 files changed, 76 insertions(+)
>>
>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>> index e1522e10e2..8efd541c4f 100644
>> --- a/xen/arch/arm/firmware/sci.c
>> +++ b/xen/arch/arm/firmware/sci.c
>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>> return 0;
>> }
>>
>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>> +{
>> + struct dt_device_node *dev;
>> + int ret = 0;
>> +
>> + switch ( domctl->cmd )
>> + {
>> + case XEN_DOMCTL_assign_device:
>> + ret = -EOPNOTSUPP;
> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
The -EOPNOTSUPP code is used because this is part of a chained call after
iommu_do_domctl, as stated in xen/common/domctl.c:859. The
XEN_DOMCTL_assign_device
call is expected to handle any DT device, regardless of whether the DT
device is
protected by an IOMMU or if the IOMMU is disabled.
The following cases are considered:
1. IOMMU Protected Device (Success)
If the device is protected by the IOMMU and iommu_do_domctl returns 0,
we continue
processing the DT device by calling sci_do_domctl.
2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
disabled,
we still proceed to call sci_do_domctl.
3. Error from iommu_do_domctl (Fail State)
If iommu_do_domctl returns any error, the system enters a fail state, and
sci_do_domctl is not called.
4. -EOPNOTSUPP from sci_do_domctl
If sci_do_domctl returns -EOPNOTSUPP, this indicates one of the following:
- The provided device is not a DT device.
- There is no cur_mediator available (indicating that the SCI subsystem
is enabled
in the configuration, but no mediator was provided).
- The current mediator does not support assign_dt_device (this is
expected to be changed;
see below for details).
In this case, -EOPNOTSUPP is returned but will be ignored, and the
original return value from iommu_do_domctl will be used as the final result.
5. Return Code from sci_do_domctl
If sci_do_domctl returns 0 (success) or an error code (failure),
the return value from iommu_do_domctl is overridden, and the result from
sci_do_domctl is returned.
Note: -EOPNOTSUPP from iommu_do_domctl will also be overridden since
step 2 was successfully completed (or failed).
>> + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT )
>> + break;
> this one
>
>> + if ( !cur_mediator )
>> + break;
> this one
>
>> + if ( !cur_mediator->assign_dt_device )
>> + break;
> and also this one? It seems more like an -EINVAL as the caller used a
> wrong parameter?
I think you are right that this case should return -EINVAL because we
should fail if mediator
without implemented mandatory features was provided. Will be fixed.
>> + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
>> + domctl->u.assign_device.u.dt.size, &dev);
>> + if ( ret )
>> + return ret;
>> +
>> + ret = sci_assign_dt_device(d, dev);
>> + if ( ret )
>> + break;
>> +
>> + break;
>> + default:
>> + /* do not fail here as call is chained with iommu handling */
> It looks like this should be an error
>
>
>> + break;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> static int __init sci_init(void)
>> {
>> struct dt_device_node *np;
>> diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
>> index 71fb54852e..b8d1bc8a62 100644
>> --- a/xen/arch/arm/include/asm/firmware/sci.h
>> +++ b/xen/arch/arm/include/asm/firmware/sci.h
>> @@ -146,6 +146,14 @@ int sci_dt_finalize(struct domain *d, void *fdt);
>> * control" functionality.
>> */
>> int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
>> +
>> +/*
>> + * SCI domctl handler
>> + *
>> + * Only XEN_DOMCTL_assign_device is handled for now.
>> + */
>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
>> #else
>>
>> static inline bool sci_domain_is_enabled(struct domain *d)
>> @@ -195,6 +203,12 @@ static inline int sci_assign_dt_device(struct domain *d,
>> return 0;
>> }
>>
>> +static inline int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>> +{
>> + return 0;
>> +}
>> +
>> #endif /* CONFIG_ARM_SCI */
>>
>> #endif /* __ASM_ARM_SCI_H */
>> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
>> index 05abb581a0..a74ee92067 100644
>> --- a/xen/common/domctl.c
>> +++ b/xen/common/domctl.c
>> @@ -27,6 +27,7 @@
>> #include <xen/vm_event.h>
>> #include <xen/monitor.h>
>> #include <asm/current.h>
>> +#include <asm/firmware/sci.h>
>> #include <asm/irq.h>
>> #include <asm/page.h>
>> #include <asm/p2m.h>
>> @@ -851,6 +852,24 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>> case XEN_DOMCTL_deassign_device:
>> case XEN_DOMCTL_get_device_group:
>> ret = iommu_do_domctl(op, d, u_domctl);
>> +
>> + if ( !ret || ret == -EOPNOTSUPP )
> It is better to invert the check:
>
> if ( ret < 0 && ret != -EOPNOTSUPP )
> return ret;
+
>> + {
>> + int ret1;
>> + /*
>> + * Add chained handling of assigned DT devices to support
>> + * access-controller functionality through SCI framework, so
>> + * DT device assign request can be passed to FW for processing and
>> + * enabling VM access to requested device.
>> + * The access-controller DT device processing is chained after IOMMU
>> + * processing and expected to be executed for any DT device
>> + * regardless if DT device is protected by IOMMU or not (or IOMMU
>> + * is disabled).
>> + */
>> + ret1 = sci_do_domctl(op, d, u_domctl);
>> + if ( ret1 != -EOPNOTSUPP )
>> + ret = ret1;
>> + }
>> break;
>>
>> case XEN_DOMCTL_get_paging_mempool_size:
>> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
>> index 075fb25a37..2624767e51 100644
>> --- a/xen/drivers/passthrough/device_tree.c
>> +++ b/xen/drivers/passthrough/device_tree.c
>> @@ -318,6 +318,12 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
>> break;
>> }
>>
>> + if ( !dt_device_is_protected(dev) )
>> + {
>> + ret = 0;
>> + break;
>> + }
> I am concerned about this: previously we would call
> iommu_assign_dt_device and the same check at the beginning of
> iommu_assign_dt_device would return -EINVAL. Now it is a success.
>
> I am not sure this is appropriate. I wonder if instead we should:
>
> - remove this chunk from the patch
> - change the return error for !dt_device_is_protected at the top of
> iommu_assign_dt_device from -EINVAL to -EOPNOTSUPP
> - this would fall into the same ret != -EOPNOTSUPP check after
> iommu_do_domctl
That's a good point. I think we should do the same for
> if ( !is_iommu_enabled(d) )
> return -EINVAL;
because in this case we should process sci as well. I will do the change
>> ret = iommu_assign_dt_device(d, dev);
>>
>> if ( ret )
>> --
>> 2.34.1
>>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-05-23 20:06 ` Stefano Stabellini
2025-06-02 7:17 ` Bertrand Marquis
2025-06-05 22:44 ` Julien Grall
@ 2025-06-12 12:03 ` Oleksii Moisieiev
2025-06-17 23:22 ` Stefano Stabellini
2 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-12 12:03 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
Hi Stefano,
I'm very sorry for a long silence. Please see my answers below:
On 23/05/2025 23:06, Stefano Stabellini wrote:
> One question for Bertrand below
>
>
> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
>> below.
>>
>> +-----------------------------------------+
>> | |
>> | EL3 TF-A SCMI |
>> +-------+--+-------+--+-------+--+-------++
>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
>> +-----+-+ +---+---+ +--+----+ +---+---+
>> smc-id0 | | | |
>> agent0 | | | |
>> +-----v--------+---------+-----------+----+
>> | | | | |
>> | | | | |
>> +--------------+---------+-----------+----+
>> smc-id1 | smc-id2| smc-idX|
>> agent1 | agent2 | agentX |
>> | | |
>> +----v---+ +--v-----+ +--v-----+
>> | | | | | |
>> | Dom0 | | Dom1 | | DomX |
>> | | | | | |
>> | | | | | |
>> +--------+ +--------+ +--------+
>>
>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
>> memory transport for every Agent in the system.
>>
>> The SCMI Agent transport channel defined by pair:
>> - smc-id: SMC/HVC id used for Doorbell
>> - shmem: shared memory for messages transfer, Xen page aligned,
>> p2m_mmio_direct_nc.
>>
>> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
>> multi-agent functionality under Xen:
>> - Xen manegement agent: trusted agents that accesses to the Base Protocol
>> commands to configure agent specific permissions
>> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
>> allowed direct HW access. At least one OSPM VM agent has to be provided
>> by FW if HW is handled only by Dom0 or Driver Domain.
>>
>> The EL3 SCMI FW expected to implement following Base protocol messages:
>> - BASE_DISCOVER_AGENT
>> - BASE_RESET_AGENT_CONFIGURATION (optional)
>> - BASE_SET_DEVICE_PERMISSIONS (optional)
>>
>> The SCI SCMI SMC multi-agent driver implements following functionality:
>> - It's initialized based on the Host DT SCMI node (only one SCMI interface
>> is supported) which describes Xen management agent SCMI interface.
>>
>> scmi_shm_0 : sram@47ff0000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff0000 0x0 0x1000>;
>> };
>> firmware {
>> scmi: scmi {
>> compatible = "arm,scmi-smc";
>> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
> some extra spaces, it might be a copy/paste error
+
>> \#address-cells = < 1>;
>> \#size-cells = < 0>;
>> \#access-controller - cells = < 1>;
>> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
>>
>> protocol@X{
>> };
>> };
>> };
>>
>> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
>> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
>>
>> chosen {
>> xen,scmi-secondary-agents = <
>> 1 0x82000003 &scmi_shm_1
>> 2 0x82000004 &scmi_shm_2
>> 3 0x82000005 &scmi_shm_3
>> 4 0x82000006 &scmi_shm_4>;
>> }
>>
>> /{
>> scmi_shm_1: sram@47ff1000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff1000 0x0 0x1000>;
>> };
>> scmi_shm_2: sram@47ff2000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff2000 0x0 0x1000>;
>> };
>> scmi_shm_3: sram@47ff3000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x47ff3000 0x0 0x1000>;
>> };
>> }
>> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
>> this agent_id.
>>
>> Note that Xen is the only one entry in the system which need to know
>> about SCMI multi-agent support.
>>
>> - It implements the SCI subsystem interface required for configuring and
>> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
>> SCMI functionality for domain it has to be configured with unique supported
>> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
>> [smc-id, shmem] defined for this SCMI Agent_id.
>> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
>> -- zero-copy, the guest domain puts SCMI message in shmem;
>> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
>> -- the Xen driver catches exception, do checks and synchronously forwards
>> it to EL3 FW.
>> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
>> management agent channel on domain destroy event. This allows to reset
>> resources used by domain and so implement use-case like domain reboot.
>>
>> Dom0 Enable SCMI SMC:
>> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
>> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
>> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
>> node according to assigned agent_id.
>>
>> Guest domains enable SCMI SMC:
>> - xl.cfg: add configuration option as below
>>
>> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
>>
>> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
>> the domain, for example:
>>
>> iomem = [
>> "47ff2,1@22001",
>> ]
> Looking at the code and the configuration options, it looks like it is
> possible to map a scmi-shmem channel at a different address for the
> guest. It seems like it would work. Is that correct?
>
Yes it will. in our case address 22001000 should be the save as
sram@22001000 in the domain device-tree.
>> - DT: add SCMI nodes to the Driver domain partial device tree as in the
>> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
>>
>> passthrough {
>> scmi_shm_0: sram@22001000 {
>> compatible = "arm,scmi-shmem";
>> reg = <0x0 0x22001000 0x0 0x1000>;
>> };
>>
>> firmware {
>> compatible = "simple-bus";
>> scmi: scmi {
>> compatible = "arm,scmi-smc";
>> arm,smc-id = <0x82000004>;
>> shmem = <&scmi_shm_0>;
>> ...
>> }
>> }
>> }
>>
>> SCMI "4.2.1.1 Device specific access control"
>>
>> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
>> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
>> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
>> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
>> to the Xen SCMI.
>>
>> &i2c1 {
>> access-controllers = <&scmi 0>;
>> };
>>
>> The Dom0 and dom0less domains DT devices will be processed automatically through
>> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
>> shell be used:
>>
>> dtdev = [
>> "/soc/i2c@e6508000",
>> ]
>>
>> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
>>
>> [1]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
>> [2]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> Thanks for the long explanation, great work! I am really looking forward
> to have this feature in the tree soon.
>
>
>> ---
>>
>> Changes in v4:
>> - toolstack comments from Anthony PERARD
>> - added dom0less support
>> - added doc for "xen,scmi-secondary-agents"
>>
>> docs/man/xl.cfg.5.pod.in | 13 +
>> docs/misc/arm/device-tree/booting.txt | 60 ++
>> docs/misc/xen-command-line.pandoc | 9 +
>> tools/libs/light/libxl_arm.c | 4 +
>> tools/libs/light/libxl_types.idl | 4 +-
>> tools/xl/xl_parse.c | 12 +
>> xen/arch/arm/dom0less-build.c | 11 +
>> xen/arch/arm/domain_build.c | 3 +-
>> xen/arch/arm/firmware/Kconfig | 11 +
>> xen/arch/arm/firmware/Makefile | 1 +
>> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
>> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
>> xen/arch/arm/firmware/scmi-shmem.h | 45 +
>> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
>> xen/include/public/arch-arm.h | 3 +
>> 15 files changed, 1371 insertions(+), 2 deletions(-)
>> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
>> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
>>
>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>> index 1ccf50b8ea..302c46d8bc 100644
>> --- a/docs/man/xl.cfg.5.pod.in
>> +++ b/docs/man/xl.cfg.5.pod.in
>> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
>> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
>> option.
>>
>> +=item B<scmi_smc_multiagent>
>> +
>> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
>> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
>> +specified for the guest.
>> +
>> =back
>>
>> +=item B<agent_id=NUMBER>
>> +
>> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
>> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
>> +existing on a single host must be unique and in the range [1..255].
>> +
>> =back
>>
>> =back
>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
>> index 8943c04173..c8923ab8b2 100644
>> --- a/docs/misc/arm/device-tree/booting.txt
>> +++ b/docs/misc/arm/device-tree/booting.txt
>> @@ -296,6 +296,20 @@ with the following properties:
>> Should be used together with dom0_scmi_smc_passthrough Xen command line
>> option.
>>
>> + - "scmi_smc_multiagent"
>> +
>> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>> + SMC calls forwarding from domain to the EL3 firmware (like ARM
>> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
>> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
>> + property.
>> +
>> +- "xen,sci_agent_id"
>> +
>> + Specifies a non-zero ARM SCI agent id for the guest. This option is
>> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
>> + the guest. The agent ids of guest must be unique and in the range [1..255].
>> +
>> Under the "xen,domain" compatible node, one or more sub-nodes are present
>> for the DomU kernel and ramdisk.
>>
>> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
>> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
>> guest physical address space. DomU1 is explicitly defined as the owner domain,
>> and DomU2 is the borrower domain.
>> +
>> +SCMI SMC multi-agent support
>> +============================
>> +
>> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
>> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
>> +according to the SCMI compliant EL3 Firmware specification with
>> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
>> +the top-level "chosen" node:
>> +
>> +- xen,scmi-secondary-agents
>> +
>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
>> + available for Xen. Each Agent defined as triple consisting of:
>> + SCMI agent_id,
>> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
>> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
>> +
>> +As an example:
>> +
>> +chosen {
>> + xen,scmi-secondary-agents = <
>> + 1 0x82000003 &scmi_shm_1
>> + 2 0x82000004 &scmi_shm_2
>> + 3 0x82000005 &scmi_shm_3
>> + 4 0x82000006 &scmi_shm_4>;
>> +}
> NIT: it should be };
+
> Looking at scmi_probe, collect_agents, and the following SCMI
> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
> information?
>
> It looks like we can discover the agend_ids for every channel, I guess
> what we need to know is the shmem location for every channel? But the
> full list of shmem channel is available below thanks to the scmi-shmem
> nodes.
>
> So, we have the list of scmi-shmem anyway, and we can probe the
> agent_id. The only parameter left is the smc_id/func_id.
>
> Or maybe smc_id/func_id can be calculated from agent_id?
>
> I am asking mostly because if a user is supposed to add this
> xen,scmi-secondary-agents property, where are they supposed to find the
> smc_id/func_id information?
>
> It is important that we write down in this document how the user is
> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
That's a very good question! The issue here is that there are no
explicit requirements defining the relationship between agent_id and
func_id.
For example, in ARM-TF, different implementations can use different
func_ids.
To provide better flexibility, we decided to separate agent_id from func_id.
Currently, the SCMI_BASE_DISCOVER_AGENT calls from the probe are intended to
verify that all registered agents are present and that the configuration
is correct.
However, I understand that this additional validation could be optional
to save traffic.
To address this, I’m considering adding a configuration option, such as
CONFIG_SCMI_AGENT_VALIDATION, which can be disabled to reduce boot time
if this
validation is not necessary for certain use cases.
Lastly, I’ll be updating the document to include clearer information
about the
relationship between func_id and agent_id in the upcoming v5.
>> +/{
>> + scmi_shm_1: sram@47ff1000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>> + };
>> + scmi_shm_2: sram@47ff2000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>> + };
>> + scmi_shm_3: sram@47ff3000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>> + };
>> + scmi_shm_3: sram@47ff4000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>> + };
> Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
> Or are under /firmware? Or are they under /chosen?
>
> I take they are under the top level node together with scmi_shm_0?
>
> Can you please also clarify in the document as well?
>
>
all these nodes are on the top level of the device-tree. But there is no
specific place for them.
They could be subnodes to some memory-region for example. I will clarify
this.
>> +}
>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
>> index 8e50f6b7c7..bc3c64d6ec 100644
>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
>> SCMI nodes removed from Dom0/hwdom device tree.
>> (for example, thin Dom0 with Driver domain use-case).
>>
>> +### dom0_scmi_agent_id (ARM)
>> +> `= <integer>`
>> +
>> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
>> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
>> +The SCMI will be disabled for Dom0 if this option is not specified
>> +(for example, thin Dom0 or dom0less use-cases).
>> +The agent ids of domains existing on a single host must be unique.
>> +
>> ### dtuart (ARM)
>> > `= path [:options]`
>>
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index 28ba9eb787..7712f53cd4 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
>> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>> break;
>> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
>> + break;
>> default:
>> LOG(ERROR, "Unknown ARM_SCI type %d",
>> d_config->b_info.arch_arm.arm_sci.type);
>> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
>> index aa2190ab5b..11e31ce786 100644
>> --- a/tools/libs/light/libxl_types.idl
>> +++ b/tools/libs/light/libxl_types.idl
>> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
>>
>> libxl_arm_sci_type = Enumeration("arm_sci_type", [
>> (0, "none"),
>> - (1, "scmi_smc")
>> + (1, "scmi_smc"),
>> + (2, "scmi_smc_multiagent")
>> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
>>
>> libxl_arm_sci = Struct("arm_sci", [
>> ("type", libxl_arm_sci_type),
>> + ("agent_id", uint8)
>> ])
>>
>> libxl_rdm_reserve = Struct("rdm_reserve", [
>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>> index bd22be9d33..81aa3797e3 100644
>> --- a/tools/xl/xl_parse.c
>> +++ b/tools/xl/xl_parse.c
>> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
>> }
>> }
>>
>> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
>> + unsigned long val = parse_ulong(oparg);
>> +
>> + if (!val || val > 255) {
>> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
>> + val);
>> + ret = ERROR_INVAL;
>> + goto parse_error;
>> + }
>> + arm_sci->agent_id = val;
>> + }
>> +
>> ptr = strtok(NULL, ",");
>> }
>>
>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>> index 0a00f03a25..43d21eb889 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>> else if ( !strcmp(sci_type, "scmi_smc") )
>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
>> + {
>> + uint32_t agent_id = 0;
>> +
>> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
>> + !agent_id )
> shouldn't we check that agent_id <= 255 ?
I see no limitation about max agent_id in DEN0056E document, it's uint32_t.
>> + return -EINVAL;
>> +
>> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>> + d_cfg->arch.arm_sci_agent_id = agent_id;
>> + }
>> else
>> {
>> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 36d28b52a4..0c9274a2b3 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
>> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
>> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
>> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
>> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
>> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
>> continue;
>>
>> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") ) diff --git a/xen/arch/arm/firmware/Kconfig
>> b/xen/arch/arm/firmware/Kconfig index 5c5f0880c4..6b051c8ada 100644
>> --- a/xen/arch/arm/firmware/Kconfig +++
>> b/xen/arch/arm/firmware/Kconfig @@ -29,6 +29,17 @@ config SCMI_SMC
>> driver domain. Use with EL3 firmware which supports only single SCMI
>> OSPM agent. +config SCMI_SMC_MA + bool "Enable ARM SCMI SMC multi-agent driver"
>> + select ARM_SCI
>> + help
>> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
>> + to EL3 firmware (TF-A) which supports multi-agent feature.
>> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
>> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
>> + allowed platform resources through dedicated SMC/HVC Shared memory based
>> + transport.
>> +
>> endchoice
>>
>> endmenu
>> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
>> index 71bdefc24a..37927e690e 100644
>> --- a/xen/arch/arm/firmware/Makefile
>> +++ b/xen/arch/arm/firmware/Makefile
>> @@ -1,2 +1,3 @@
>> obj-$(CONFIG_ARM_SCI) += sci.o
>> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
>> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
>> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
>> new file mode 100644
>> index 0000000000..3f4b9c5d6b
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-proto.h
>> @@ -0,0 +1,164 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Arm System Control and Management Interface definitions
>> + * Version 3.0 (DEN0056C)
>> + *
>> + * Copyright (c) 2024 EPAM Systems
>> + */
>> +
>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> NIT: ARM_FIRMWARE_SCMI_PROTO_H
+
>> +#include <xen/stdint.h>
>> +
>> +#define SCMI_SHORT_NAME_MAX_SIZE 16
>> +
>> +/* SCMI status codes. See section 4.1.4 */
>> +#define SCMI_SUCCESS 0
>> +#define SCMI_NOT_SUPPORTED (-1)
>> +#define SCMI_INVALID_PARAMETERS (-2)
>> +#define SCMI_DENIED (-3)
>> +#define SCMI_NOT_FOUND (-4)
>> +#define SCMI_OUT_OF_RANGE (-5)
>> +#define SCMI_BUSY (-6)
>> +#define SCMI_COMMS_ERROR (-7)
>> +#define SCMI_GENERIC_ERROR (-8)
>> +#define SCMI_HARDWARE_ERROR (-9)
>> +#define SCMI_PROTOCOL_ERROR (-10)
>> +
>> +/* Protocol IDs */
>> +#define SCMI_BASE_PROTOCOL 0x10
>> +
>> +/* Base protocol message IDs */
>> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
>> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
>> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
>> +#define SCMI_BASE_DISCOVER_AGENT 0x7
>> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
>> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
>> +
>> +typedef struct scmi_msg_header {
>> + uint8_t id;
>> + uint8_t type;
>> + uint8_t protocol;
>> + uint32_t status;
>> +} scmi_msg_header_t;
>> +
>> +/* Table 2 Message header format */
>> +#define SCMI_HDR_ID GENMASK(7, 0)
>> +#define SCMI_HDR_TYPE GENMASK(9, 8)
>> +#define SCMI_HDR_PROTO GENMASK(17, 10)
>> +
>> +#define SCMI_FIELD_GET(_mask, _reg) \
>> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
>> +#define SCMI_FIELD_PREP(_mask, _val) \
>> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
>> +
>> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
>> +{
>> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
>> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
>> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
>> +}
>> +
>> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
>> +{
>> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
>> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
>> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
>> +}
>> +
>> +static inline int scmi_to_xen_errno(int scmi_status)
>> +{
>> + if ( scmi_status == SCMI_SUCCESS )
>> + return 0;
>> +
>> + switch ( scmi_status )
>> + {
>> + case SCMI_NOT_SUPPORTED:
>> + return -EOPNOTSUPP;
>> + case SCMI_INVALID_PARAMETERS:
>> + return -EINVAL;
>> + case SCMI_DENIED:
>> + return -EACCES;
>> + case SCMI_NOT_FOUND:
>> + return -ENOENT;
>> + case SCMI_OUT_OF_RANGE:
>> + return -ERANGE;
>> + case SCMI_BUSY:
>> + return -EBUSY;
>> + case SCMI_COMMS_ERROR:
>> + return -ENOTCONN;
>> + case SCMI_GENERIC_ERROR:
>> + return -EIO;
>> + case SCMI_HARDWARE_ERROR:
>> + return -ENXIO;
>> + case SCMI_PROTOCOL_ERROR:
>> + return -EBADMSG;
>> + default:
>> + return -EINVAL;
>> + }
>> +}
>> +
>> +/* PROTOCOL_VERSION */
>> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
>> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
>> +
>> +struct scmi_msg_prot_version_p2a {
>> + uint32_t version;
>> +} __packed;
>> +
>> +/* BASE PROTOCOL_ATTRIBUTES */
>> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
>> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
>> +
>> +struct scmi_msg_base_attributes_p2a {
>> + uint32_t attributes;
>> +} __packed;
>> +
>> +/*
>> + * BASE_DISCOVER_AGENT
>> + */
>> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
>> +
>> +struct scmi_msg_base_discover_agent_a2p {
>> + uint32_t agent_id;
>> +} __packed;
>> +
>> +struct scmi_msg_base_discover_agent_p2a {
>> + uint32_t agent_id;
>> + char name[SCMI_SHORT_NAME_MAX_SIZE];
>> +} __packed;
>> +
>> +/*
>> + * BASE_SET_DEVICE_PERMISSIONS
>> + */
>> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
>> +
>> +struct scmi_msg_base_set_device_permissions_a2p {
>> + uint32_t agent_id;
>> + uint32_t device_id;
>> + uint32_t flags;
>> +} __packed;
>> +
>> +/*
>> + * BASE_RESET_AGENT_CONFIGURATION
>> + */
>> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
>> +
>> +struct scmi_msg_base_reset_agent_cfg_a2p {
>> + uint32_t agent_id;
>> + uint32_t flags;
>> +} __packed;
>> +
>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
>> new file mode 100644
>> index 0000000000..dd613ee0b5
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-shmem.c
>> @@ -0,0 +1,173 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>> + *
>> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>> + * Copyright (c) 2025 EPAM Systems
>> + */
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +
>> +#include <asm/io.h>
>> +#include <xen/err.h>
>> +
>> +#include "scmi-proto.h"
>> +#include "scmi-shmem.h"
> This code is written more generically than the description implies. If
> we only want to make SMC calls to TF-A on EL3 and exchange data with it
> over shared memory, then I think:
> - we don't need the __iomem tag, as there is no MMIO
> - we only need a DMB, not a DSB (readl and writel imply DSB, use only
> readl_relaxed and writel_relaxed)
>
> On the other hand, if we also want to handle the case where the SCMI
> server could be on a separate co-processor, then what this code is doing
> is not sufficient because we also need a dcache flush, in addition to
> the DSB.
>
> Bertrand, can you double-check?
>
>
>> +/*
>> + * Copy data from IO memory space to "real" memory space.
>> + */
>> +static void __memcpy_fromio(void *to, const volatile void __iomem *from,
>> + size_t count)
>> +{
>> + while ( count && !IS_ALIGNED((unsigned long)from, 4) )
>> + {
>> + *(u8 *)to = readb_relaxed(from);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +
>> + while ( count >= 4 )
>> + {
>> + *(u32 *)to = readl_relaxed(from);
>> + from += 4;
>> + to += 4;
>> + count -= 4;
>> + }
>> +
>> + while ( count )
>> + {
>> + *(u8 *)to = readb_relaxed(from);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +}
>> +
>> +/*
>> + * Copy data from "real" memory space to IO memory space.
>> + */
>> +static void __memcpy_toio(volatile void __iomem *to, const void *from,
>> + size_t count)
>> +{
>> + while ( count && !IS_ALIGNED((unsigned long)to, 4) )
>> + {
>> + writeb_relaxed(*(u8 *)from, to);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +
>> + while ( count >= 4 )
>> + {
>> + writel_relaxed(*(u32 *)from, to);
>> + from += 4;
>> + to += 4;
>> + count -= 4;
>> + }
>> +
>> + while ( count )
>> + {
>> + writeb_relaxed(*(u8 *)from, to);
>> + from++;
>> + to++;
>> + count--;
>> + }
>> +}
> I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
> we use a simple memcpy?
This approach was used because we're trying to access shared memory
between two independent systems:
Arm-TF and Xen in our case which places some chunks of data to the same
memory. And, according to the [0]
```
Some devices (such as framebuffers) would like to use larger transfers than
8 bytes at a time. For these devices, the memcpy_toio(),
memcpy_fromio() and memset_io() functions are
provided. Do not use memset or memcpy on IO addresses; they are not
guaranteed to copy data in order.
```
Also, the same approach was used by Arm team when introducing scmi
driver to the Linux kernel [1]
[0]: https://www.kernel.org/doc/Documentation/driver-api/device-io.rst
[1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
>> +static inline int
>> +shmem_channel_is_free(const volatile struct scmi_shared_mem __iomem *shmem)
>> +{
>> + return (readl(&shmem->channel_status) &
>> + SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE) ? 0 : -EBUSY;
>> +}
>> +
>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len)
>> +{
>> + int ret;
>> +
>> + if ( (len + sizeof(shmem->msg_header)) > SCMI_SHMEM_MAPPED_SIZE )
>> + {
>> + printk(XENLOG_ERR "scmi: Wrong size of smc message. Data is invalid\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = shmem_channel_is_free(shmem);
>> + if ( ret )
>> + return ret;
>> +
>> + writel_relaxed(0x0, &shmem->channel_status);
>> + /* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
>> + writel_relaxed(0x0, &shmem->flags);
>> + writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
>> + writel(pack_scmi_header(hdr), &shmem->msg_header);
>> +
>> + if ( len > 0 && data )
>> + __memcpy_toio(shmem->msg_payload, data, len);
>> +
>> + return 0;
>> +}
>> +
>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len)
>> +{
>> + int recv_len;
>> + int ret;
>> + int pad = sizeof(hdr->status);
>> +
>> + if ( len >= SCMI_SHMEM_MAPPED_SIZE - sizeof(shmem) )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Wrong size of input smc message. Data may be invalid\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = shmem_channel_is_free(shmem);
>> + if ( ret )
>> + return ret;
>> +
>> + recv_len = readl(&shmem->length) - sizeof(shmem->msg_header);
>> +
>> + if ( recv_len < 0 )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Wrong size of smc message. Data may be invalid\n");
>> + return -EINVAL;
>> + }
>> +
>> + unpack_scmi_header(readl(&shmem->msg_header), hdr);
>> +
>> + hdr->status = readl(&shmem->msg_payload);
>> + recv_len = recv_len > pad ? recv_len - pad : 0;
>> +
>> + ret = scmi_to_xen_errno(hdr->status);
>> + if ( ret )
>> + {
>> + printk(XENLOG_DEBUG "scmi: Error received: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + if ( recv_len > len )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Not enough buffer for message %d, expecting %d\n",
>> + recv_len, len);
>> + return -EINVAL;
>> + }
>> +
>> + if ( recv_len > 0 )
>> + __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
>> +
>> + return 0;
>> +}
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/arch/arm/firmware/scmi-shmem.h b/xen/arch/arm/firmware/scmi-shmem.h
>> new file mode 100644
>> index 0000000000..2f8e23ff76
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-shmem.h
>> @@ -0,0 +1,45 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Arm System Control and Management Interface definitions
>> + * Version 3.0 (DEN0056C)
>> + * Shared Memory based Transport
>> + *
>> + * Copyright (c) 2024 EPAM Systems
>> + */
>> +
>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
>> +#define XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
> NIT: ARM_FIRMWARE_SCMI_SHMEM_H
>
+
>> +#include <xen/stdint.h>
>> +
>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE BIT(0, UL)
>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_ERROR BIT(1, UL)
>> +
>> +struct scmi_shared_mem {
>> + uint32_t reserved;
>> + uint32_t channel_status;
>> + uint32_t reserved1[2];
>> + uint32_t flags;
>> + uint32_t length;
>> + uint32_t msg_header;
>> + uint8_t msg_payload[];
>> +};
>> +
>> +#define SCMI_SHMEM_MAPPED_SIZE PAGE_SIZE
>> +
>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len);
>> +
>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>> + scmi_msg_header_t *hdr, void *data, int len);
>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_ */
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/arch/arm/firmware/scmi-smc-multiagent.c b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>> new file mode 100644
>> index 0000000000..e023bca3a1
>> --- /dev/null
>> +++ b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>> @@ -0,0 +1,860 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>> + *
>> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>> + * Copyright (c) 2025 EPAM Systems
>> + */
>> +
>> +#include <xen/acpi.h>
>> +
>> +#include <xen/device_tree.h>
>> +#include <xen/init.h>
>> +#include <xen/iocap.h>
>> +#include <xen/err.h>
>> +#include <xen/libfdt/libfdt.h>
>> +#include <xen/param.h>
>> +#include <xen/sched.h>
>> +#include <xen/vmap.h>
>> +
>> +#include <asm/firmware/sci.h>
>> +#include <asm/smccc.h>
>> +
>> +#include "scmi-proto.h"
>> +#include "scmi-shmem.h"
>> +
>> +#define SCMI_AGENT_ID_INVALID 0xFF
>> +
>> +static uint8_t __initdata opt_dom0_scmi_agent_id = SCMI_AGENT_ID_INVALID;
>> +integer_param("dom0_scmi_agent_id", opt_dom0_scmi_agent_id);
>> +
>> +#define SCMI_SECONDARY_AGENTS "xen,scmi-secondary-agents"
>> +
>> +#define HYP_CHANNEL 0x0
>> +
>> +struct scmi_channel {
>> + uint32_t agent_id;
>> + uint32_t func_id;
>> + domid_t domain_id;
>> + uint64_t paddr;
>> + uint64_t len;
>> + struct scmi_shared_mem __iomem *shmem;
>> + spinlock_t lock;
>> + struct list_head list;
>> +};
>> +
>> +struct scmi_data {
>> + struct list_head channel_list;
>> + spinlock_t channel_list_lock;
>> + uint32_t func_id;
>> + bool initialized;
>> + uint32_t shmem_phandle;
>> + struct dt_device_node *dt_dev;
>> +};
>> +
>> +static struct scmi_data scmi_data;
>> +
>> +static int send_smc_message(struct scmi_channel *chan_info,
>> + scmi_msg_header_t *hdr, void *data, int len)
>> +{
>> + struct arm_smccc_res resp;
>> + int ret;
>> +
>> + ret = shmem_put_message(chan_info->shmem, hdr, data, len);
>> + if ( ret )
>> + return ret;
>> +
>> + arm_smccc_1_1_smc(chan_info->func_id, 0, 0, 0, 0, 0, 0, 0, &resp);
>> +
>> + if ( resp.a0 )
>> + return -EOPNOTSUPP;
> Why if repo.a0 != 0 then we assume -EOPNOTSUPP? Is this part of the SCMI
> specification?
Please see 7.1 of [1]. The following codes are supported :
0 for success
-1 as not_supported
-2 as not_required
-3 as invalid_parameter.
For all non-zero results we assume that this call is not supported by
system or has different revision
which we also doesn't support.
[1] DEN0028
https://documentation-service.arm.com/static/5f8edaeff86e16515cdbe4c6
>> + return 0;
>> +}
>> +
>> +static int do_smc_xfer(struct scmi_channel *chan_info, scmi_msg_header_t *hdr,
>> + void *tx_data, int tx_size, void *rx_data, int rx_size)
>> +{
>> + int ret = 0;
>> +
>> + ASSERT(chan_info && chan_info->shmem);
>> +
>> + if ( !hdr )
>> + return -EINVAL;
>> +
>> + spin_lock(&chan_info->lock);
>> +
>> + printk(XENLOG_DEBUG
>> + "scmi: agent_id = %d msg_id = %x type = %d, proto = %x\n",
>> + chan_info->agent_id, hdr->id, hdr->type, hdr->protocol);
>> +
>> + ret = send_smc_message(chan_info, hdr, tx_data, tx_size);
>> + if ( ret )
>> + goto clean;
>> +
>> + ret = shmem_get_response(chan_info->shmem, hdr, rx_data, rx_size);
>> +
>> +clean:
>> + printk(XENLOG_DEBUG
>> + "scmi: get smc response agent_id = %d msg_id = %x proto = %x res=%d\n",
>> + chan_info->agent_id, hdr->id, hdr->protocol, ret);
>> +
>> + spin_unlock(&chan_info->lock);
>> +
>> + return ret;
>> +}
>> +
>> +static struct scmi_channel *get_channel_by_id(uint32_t agent_id)
>> +{
>> + struct scmi_channel *curr;
>> + bool found = false;
>> +
>> + spin_lock(&scmi_data.channel_list_lock);
>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>> + {
>> + if ( curr->agent_id == agent_id )
>> + {
>> + found = true;
>> + break;
>> + }
>> + }
>> +
>> + spin_unlock(&scmi_data.channel_list_lock);
>> + if ( found )
>> + return curr;
>> +
>> + return NULL;
>> +}
>> +
>> +static struct scmi_channel *acquire_scmi_channel(struct domain *d,
>> + uint32_t agent_id)
>> +{
>> + struct scmi_channel *curr;
>> + struct scmi_channel *ret = ERR_PTR(-ENOENT);
>> +
>> + spin_lock(&scmi_data.channel_list_lock);
>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>> + {
>> + if ( curr->agent_id == agent_id )
>> + {
>> + if ( curr->domain_id != DOMID_INVALID )
>> + {
>> + ret = ERR_PTR(-EEXIST);
>> + break;
>> + }
>> +
>> + curr->domain_id = d->domain_id;
>> + ret = curr;
>> + break;
>> + }
>> + }
>> +
>> + spin_unlock(&scmi_data.channel_list_lock);
>> +
>> + return ret;
>> +}
>> +
>> +static void relinquish_scmi_channel(struct scmi_channel *channel)
>> +{
>> + ASSERT(channel != NULL);
>> +
>> + spin_lock(&scmi_data.channel_list_lock);
>> + channel->domain_id = DOMID_INVALID;
>> + spin_unlock(&scmi_data.channel_list_lock);
>> +}
>> +
>> +static int map_channel_memory(struct scmi_channel *channel)
>> +{
>> + ASSERT(channel && channel->paddr);
>> + channel->shmem = ioremap_nocache(channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
> ioremap is for MMIO, if these shared memory channels are on DDR, then it
> would not be the right call. Are the "arm,scmi-shmem" address ranges
> part of the memory node ranges? Or are they completely separate?
>
> Also, why nocache? Wouldn't we want ioremap_cache?
>
Currently, they are separate nodes but could potentially be part of the
"memory-region".
From what I see in the Linux kernel, devm_ioremap is being used to map
the memory.
Could there be something more to this approach?
[0]:https://git.iliana.fyi/linux/patch/?id=1dc6558062dadfabd2fb3bd885fa6e92ec7196f2
>> + if ( !channel->shmem )
>> + return -ENOMEM;
>> +
>> + channel->shmem->channel_status = SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE;
>> + printk(XENLOG_DEBUG "scmi: Got shmem %lx after vmap %p\n", channel->paddr,
>> + channel->shmem);
>> +
>> + return 0;
>> +}
>> +
>> +static void unmap_channel_memory(struct scmi_channel *channel)
>> +{
>> + ASSERT(channel && channel->shmem);
>> + iounmap(channel->shmem);
>> + channel->shmem = NULL;
>> +}
>> +
>> +static struct scmi_channel *smc_create_channel(uint32_t agent_id,
>> + uint32_t func_id, uint64_t addr)
>> +{
>> + struct scmi_channel *channel;
>> +
>> + channel = get_channel_by_id(agent_id);
>> + if ( channel )
>> + return ERR_PTR(EEXIST);
>> +
>> + channel = xmalloc(struct scmi_channel);
>> + if ( !channel )
>> + return ERR_PTR(ENOMEM);
>> +
>> + spin_lock_init(&channel->lock);
>> + channel->agent_id = agent_id;
>> + channel->func_id = func_id;
>> + channel->domain_id = DOMID_INVALID;
>> + channel->shmem = NULL;
>> + channel->paddr = addr;
>> + list_add_tail(&channel->list, &scmi_data.channel_list);
>> + return channel;
>> +}
>> +
>> +static void free_channel_list(void)
>> +{
>> + struct scmi_channel *curr, *_curr;
>> +
>> + list_for_each_entry_safe(curr, _curr, &scmi_data.channel_list, list)
>> + {
>> + list_del(&curr->list);
>> + xfree(curr);
>> + }
>> +}
>> +
>> +static int __init
>> +scmi_dt_read_hyp_channel_addr(struct dt_device_node *scmi_node, u64 *addr,
>> + u64 *size)
>> +{
>> + struct dt_device_node *shmem_node;
>> + const __be32 *prop;
>> +
>> + prop = dt_get_property(scmi_node, "shmem", NULL);
>> + if ( !prop )
>> + return -EINVAL;
>> +
>> + shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
>> + if ( IS_ERR_OR_NULL(shmem_node) )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Device tree error, can't parse reserved memory %ld\n",
>> + PTR_ERR(shmem_node));
>> + return PTR_ERR(shmem_node);
>> + }
>> +
>> + return dt_device_get_address(shmem_node, 0, addr, size);
>> +}
>> +
>> +/*
>> + * Handle Dom0 SCMI specific DT nodes
>> + *
>> + * Make a decision on copying SCMI specific nodes into Dom0 device tree.
>> + * For SCMI multi-agent case:
>> + * - shmem nodes will not be copied and generated instead if SCMI
>> + * is enabled for Dom0
>> + * - scmi node will be copied if SCMI is enabled for Dom0
>> + */
>> +static bool scmi_dt_handle_node(struct domain *d, struct dt_device_node *node)
>> +{
>> + static const struct dt_device_match skip_matches[] __initconst = {
>> + DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
>> + { /* sentinel */ },
>> + };
>> + static const struct dt_device_match scmi_matches[] __initconst = {
>> + DT_MATCH_PATH("/firmware/scmi"),
>> + { /* sentinel */ },
>> + };
>> +
>> + if ( !scmi_data.initialized )
>> + return false;
>> +
>> + /* always drop shmem */
>> + if ( dt_match_node(skip_matches, node) )
>> + {
>> + dt_dprintk(" Skip scmi shmem\n");
>> + return true;
>> + }
>> +
>> + /* drop scmi if not enabled */
>> + if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
>> + {
>> + dt_dprintk(" Skip scmi node\n");
>> + return true;
>> + }
>> +
>> + return false;
>> +}
>> +
>> +/*
>> + * Finalize Dom0 SCMI specific DT nodes
>> + *
>> + * if SCMI is enabled for Dom0:
>> + * - generate shmem node
>> + * - map SCMI shmem MMIO into Dom0
>> + */
>> +static int scmi_dt_finalize(struct domain *d, void *fdt)
>> +{
>> + __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
>> + struct scmi_channel *channel;
>> + int nodeoffset;
>> + __be32 *cells;
>> + __be32 val;
>> + char buf[64];
>> + int res, rc;
>> +
>> + if ( !sci_domain_is_enabled(d) )
>> + return 0;
>> +
>> + channel = d->arch.sci_data;
>> +
>> + /*
>> + * Replace "arm,smc-id" with proper value assigned for Dom0 SCMI channel
>> + */
>> + nodeoffset = fdt_node_offset_by_compatible(fdt, -1, "arm,scmi-smc");
>> + if ( nodeoffset < 0 )
>> + return -ENODEV;
>> +
>> + cells = (__be32 *)&val;
>> + dt_set_cell(&cells, 1, channel->func_id);
>> + res = fdt_setprop_inplace(fdt, nodeoffset, "arm,smc-id", &val, sizeof(val));
>> + if ( res )
>> + return -EINVAL;
>> +
> Are you sure it is worth to go through all this trouble to modify FDT in
> place when we could simply generate the DT node from scratch like we do
> for example for the GIC? This seems to be more error prone as well. Is
> generating it from scratch is really difficult? If it is difficult then OK.
>
In the last patch [0] of the series, there is a proposal to separate
the Xen privileged agent from the Dom0 agent. This eliminates the
need to modify the Xen Device Tree Blob (DTB), which is a positive
improvement
since the Dom0 agent node is already present in the device tree.
However, the issue with SCMI node generation lies in the need to handle
the list of protocols that are supported across the system.
If we want to generate the SCMI node for Dom0, we need to:
Copy these protocols from a centralized or predefined source.
Set the correct phandle for each subnode within the SCMI node, ensuring
accurate representation and functionality.
This extra step of managing the protocols and phandles adds
complexity but is necessary for ensuring proper support for Dom0.
[0]
https://lists.xenproject.org/archives/html/xen-devel/2025-05/msg01041.html
>> + /*
>> + * All SCMI shmem nodes should be removed from Dom0 DT at this point, so
>> + * the shmem node for Dom0 need to be generated from SCMI channel assigned
>> + * to Dom0.
>> + * The original SCMI shmem node from platform DT is used by Xen SCMI driver
>> + * itself as privileged channel (agent_id=0) to manage other SCMI
>> + * agents (domains).
>> + */
>> + snprintf(buf, sizeof(buf), "scmi-shmem@%lx", channel->paddr);
>> +
>> + res = fdt_begin_node(fdt, buf);
>> + if ( res )
>> + return res;
>> +
>> + res = fdt_property_string(fdt, "compatible", "arm,scmi-shmem");
>> + if ( res )
>> + return res;
>> +
>> + cells = ®[0];
>> +
>> + dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>> + channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
>> +
>> + res = fdt_property(fdt, "reg", reg, sizeof(reg));
>> + if ( res )
>> + return res;
>> +
>> + res = fdt_property_cell(fdt, "phandle", scmi_data.shmem_phandle);
>> + if ( res )
>> + return res;
>> +
>> + res = fdt_end_node(fdt);
>> + if ( res )
>> + return res;
>> +
>> + /*
>> + * Map SCMI shmem into Dom0 here as shmem nodes are excluded from
>> + * generic Dom0 DT processing
>> + */
>> + res = iomem_permit_access(d, paddr_to_pfn(channel->paddr),
>> + paddr_to_pfn(channel->paddr +
>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>> + if ( res )
>> + return res;
>> +
>> + res = map_regions_p2mt(d, gaddr_to_gfn(channel->paddr),
>> + PFN_UP(SCMI_SHMEM_MAPPED_SIZE),
>> + maddr_to_mfn(channel->paddr), p2m_mmio_direct_nc);
>> + if ( res )
>> + {
>> + rc = iomem_deny_access(d, paddr_to_pfn(channel->paddr),
>> + paddr_to_pfn(channel->paddr +
>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>> + if ( rc )
>> + printk(XENLOG_ERR "scmi: Unable to deny iomem access , err = %d\n",
>> + rc);
>> + }
>> +
>> + return res;
>> +}
>> +
>> +static int scmi_assign_device(uint32_t agent_id, uint32_t device_id,
>> + uint32_t flags)
>> +{
>> + struct scmi_msg_base_set_device_permissions_a2p tx;
>> + struct scmi_channel *channel;
>> + scmi_msg_header_t hdr;
>> + int ret;
>> +
>> + channel = get_channel_by_id(HYP_CHANNEL);
>> + if ( !channel )
>> + return -EINVAL;
>> +
>> + hdr.id = SCMI_BASE_SET_DEVICE_PERMISSIONS;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + tx.agent_id = agent_id;
>> + tx.device_id = device_id;
>> + tx.flags = flags;
>> +
>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>> + if ( ret == -EOPNOTSUPP )
>> + return 0;
> Is it actually OK to pretend that everything worked if the return is
> -EOPNOTSUPP? I mean that in this case can we assume that the device is
> actually assigned anyway? Wouldn't follow up SCMI operations on this
> device fail?
>
I think you right. Will fix in v5
>> + return ret;
>> +}
>> +
>> +static int scmi_dt_assign_device(struct domain *d,
>> + struct dt_phandle_args *ac_spec)
>> +{
>> + struct scmi_channel *agent_channel;
>> + uint32_t scmi_device_id = ac_spec->args[0];
>> + int ret;
>> +
>> + if ( !d->arch.sci_data )
>> + return 0;
>> +
>> + /* The access-controllers is specified for DT dev, but it's not a SCMI */
>> + if ( ac_spec->np != scmi_data.dt_dev )
>> + return 0;
> I wonder if this should be an error
>
We don’t expect the DT node access-controller to rely solely on SCMI.
Therefore,
if a DT node has an access-controller but it’s not associated with SCMI,
we should simply
ignore that node without raising an error.
That’s my understanding of the situation.
>> + agent_channel = d->arch.sci_data;
>> +
>> + spin_lock(&agent_channel->lock);
>> +
>> + ret = scmi_assign_device(agent_channel->agent_id, scmi_device_id,
>> + SCMI_BASE_DEVICE_ACCESS_ALLOW);
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: could not assign dev for %pd agent:%d dev_id:%u (%d)",
>> + d, agent_channel->agent_id, scmi_device_id, ret);
>> + }
>> +
>> + spin_unlock(&agent_channel->lock);
>> + return ret;
>> +}
>> +
>> +static __init int collect_agents(struct dt_device_node *scmi_node)
>> +{
>> + const struct dt_device_node *chosen_node;
>> + const __be32 *prop;
>> + uint32_t len, i;
>> +
>> + chosen_node = dt_find_node_by_path("/chosen");
>> + if ( !chosen_node )
>> + {
>> + printk(XENLOG_ERR "scmi: chosen node not found\n");
>> + return -ENOENT;
>> + }
>> +
>> + prop = dt_get_property(chosen_node, SCMI_SECONDARY_AGENTS, &len);
>> + if ( !prop )
>> + {
>> + printk(XENLOG_WARNING "scmi: No %s property found\n",
>> + SCMI_SECONDARY_AGENTS);
>> + return -ENODEV;
>> + }
>> +
>> + if ( len % (3 * sizeof(uint32_t)) )
>> + {
>> + printk(XENLOG_ERR "scmi: Invalid length of %s property: %d\n",
>> + SCMI_SECONDARY_AGENTS, len);
>> + return -EINVAL;
>> + }
>> +
>> + for ( i = 0; i < len / (3 * sizeof(uint32_t)); i++ )
>> + {
>> + uint32_t agent_id = be32_to_cpu(*prop++);
>> + uint32_t smc_id = be32_to_cpu(*prop++);
>> + uint32_t shmem_phandle = be32_to_cpu(*prop++);
>> + struct dt_device_node *node = dt_find_node_by_phandle(shmem_phandle);
>> + u64 addr, size;
>> + int ret;
>> +
>> + if ( !node )
>> + {
>> + printk(XENLOG_ERR "scmi: Could not find shmem node for agent %d\n",
>> + agent_id);
>> + return -EINVAL;
>> + }
>> +
>> + ret = dt_device_get_address(node, 0, &addr, &size);
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Could not read shmem address for agent %d: %d",
>> + agent_id, ret);
>> + return ret;
>> + }
>> +
>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>> + {
>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = PTR_RET(smc_create_channel(agent_id, smc_id, addr));
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR "scmi: Could not create channel for agent %d: %d",
>> + agent_id, ret);
>> + return ret;
>> + }
>> +
>> + printk(XENLOG_DEBUG "scmi: Agent %d SMC %X addr %lx\n", agent_id,
>> + smc_id, addr);
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int scmi_domain_init(struct domain *d,
>> + struct xen_domctl_createdomain *config)
>> +{
>> + struct scmi_channel *channel;
>> + int ret;
>> +
>> + if ( !scmi_data.initialized )
>> + return 0;
>> +
>> + /*
>> + * Special case for Dom0 - the SCMI support is enabled basing on
>> + * "dom0_sci_agent_id" Xen command line parameter
>> + */
>> + if ( is_hardware_domain(d) )
>> + {
>> + if ( opt_dom0_scmi_agent_id != SCMI_AGENT_ID_INVALID )
>> + {
>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>> + config->arch.arm_sci_agent_id = opt_dom0_scmi_agent_id;
>> + }
>> + else
>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>> + }
>> +
>> + if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE )
>> + return 0;
>> +
>> + channel = acquire_scmi_channel(d, config->arch.arm_sci_agent_id);
>> + if ( IS_ERR(channel) )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Failed to acquire SCMI channel for agent_id %u: %ld\n",
>> + config->arch.arm_sci_agent_id, PTR_ERR(channel));
>> + return PTR_ERR(channel);
>> + }
>> +
>> + printk(XENLOG_INFO
>> + "scmi: Acquire channel id = 0x%x, domain_id = %d paddr = 0x%lx\n",
>> + channel->agent_id, channel->domain_id, channel->paddr);
>> +
>> + /*
>> + * Dom0 (if present) needs to have an access to the guest memory range
>> + * to satisfy iomem_access_permitted() check in XEN_DOMCTL_iomem_permission
>> + * domctl.
> Ideally this should not be needed but I understand we don't have an
> easy solution, I think we can go ahead with this for now.
>
>> + */
>> + if ( hardware_domain && !is_hardware_domain(d) )
>> + {
>> + ret = iomem_permit_access(hardware_domain, paddr_to_pfn(channel->paddr),
>> + paddr_to_pfn(channel->paddr + PAGE_SIZE - 1));
>> + if ( ret )
>> + goto error;
>> + }
>> +
>> + d->arch.sci_data = channel;
>> + d->arch.sci_enabled = true;
>> +
>> + return 0;
>> +
>> +error:
>> + relinquish_scmi_channel(channel);
>> + return ret;
>> +}
>> +
>> +int scmi_domain_sanitise_config(struct xen_domctl_createdomain *config)
>> +{
>> + if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
>> + config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA )
>> + {
>> + dprintk(XENLOG_INFO, "scmi: Unsupported ARM_SCI type\n");
>> + return -EINVAL;
>> + }
>> + else if ( config->arch.arm_sci_type ==
>> + XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA &&
>> + config->arch.arm_sci_agent_id == 0 )
>> + {
>> + dprintk(XENLOG_INFO,
>> + "scmi: A zero ARM SCMI agent_id is not supported\n");
>> + return -EINVAL;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int scmi_relinquish_resources(struct domain *d)
>> +{
>> + int ret;
>> + struct scmi_channel *channel, *agent_channel;
>> + scmi_msg_header_t hdr;
>> + struct scmi_msg_base_reset_agent_cfg_a2p tx;
>> +
>> + if ( !d->arch.sci_data )
>> + return 0;
>> +
>> + agent_channel = d->arch.sci_data;
>> +
>> + spin_lock(&agent_channel->lock);
>> + tx.agent_id = agent_channel->agent_id;
>> + spin_unlock(&agent_channel->lock);
>> +
>> + channel = get_channel_by_id(HYP_CHANNEL);
>> + if ( !channel )
>> + {
>> + printk(XENLOG_ERR
>> + "scmi: Unable to get Hypervisor scmi channel for domain %d\n",
>> + d->domain_id);
>> + return -EINVAL;
>> + }
>> +
>> + hdr.id = SCMI_BASE_RESET_AGENT_CONFIGURATION;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + tx.flags = 0;
>> +
>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>> + if ( ret == -EOPNOTSUPP )
>> + return 0;
>> +
>> + return ret;
>> +}
>> +
>> +static void scmi_domain_destroy(struct domain *d)
>> +{
>> + struct scmi_channel *channel;
>> +
>> + if ( !d->arch.sci_data )
>> + return;
>> +
>> + channel = d->arch.sci_data;
>> + spin_lock(&channel->lock);
>> +
>> + relinquish_scmi_channel(channel);
>> + printk(XENLOG_DEBUG "scmi: Free domain %d\n", d->domain_id);
>> +
>> + d->arch.sci_data = NULL;
>> + d->arch.sci_enabled = true;
>> +
>> + spin_unlock(&channel->lock);
>> +}
>> +
>> +static bool scmi_handle_call(struct cpu_user_regs *regs)
>> +{
>> + uint32_t fid = (uint32_t)get_user_reg(regs, 0);
>> + struct scmi_channel *agent_channel;
>> + struct domain *d = current->domain;
>> + struct arm_smccc_res resp;
>> + bool res = false;
>> +
>> + if ( !sci_domain_is_enabled(d) )
>> + return false;
>> +
>> + agent_channel = d->arch.sci_data;
>> + spin_lock(&agent_channel->lock);
>> +
>> + if ( agent_channel->func_id != fid )
>> + {
>> + res = false;
>> + goto unlock;
>> + }
>> +
>> + arm_smccc_1_1_smc(fid,
>> + get_user_reg(regs, 1),
>> + get_user_reg(regs, 2),
>> + get_user_reg(regs, 3),
>> + get_user_reg(regs, 4),
>> + get_user_reg(regs, 5),
>> + get_user_reg(regs, 6),
>> + get_user_reg(regs, 7),
>> + &resp);
>> +
>> + set_user_reg(regs, 0, resp.a0);
>> + set_user_reg(regs, 1, resp.a1);
>> + set_user_reg(regs, 2, resp.a2);
>> + set_user_reg(regs, 3, resp.a3);
>> + res = true;
>> +unlock:
>> + spin_unlock(&agent_channel->lock);
>> +
>> + return res;
>> +}
>> +
>> +static const struct sci_mediator_ops scmi_ops = {
>> + .domain_init = scmi_domain_init,
>> + .domain_destroy = scmi_domain_destroy,
>> + .relinquish_resources = scmi_relinquish_resources,
>> + .handle_call = scmi_handle_call,
>> + .dom0_dt_handle_node = scmi_dt_handle_node,
>> + .dom0_dt_finalize = scmi_dt_finalize,
>> + .domain_sanitise_config = scmi_domain_sanitise_config,
>> + .assign_dt_device = scmi_dt_assign_device,
>> +};
>> +
>> +static int __init scmi_check_smccc_ver(void)
>> +{
>> + if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
>> + {
>> + printk(XENLOG_WARNING
>> + "scmi: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
>> + return -ENOSYS;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static __init int scmi_probe(struct dt_device_node *scmi_node, const void *data)
>> +{
>> + u64 addr, size;
>> + int ret, i;
>> + struct scmi_channel *channel, *agent_channel;
>> + int n_agents;
>> + scmi_msg_header_t hdr;
>> + struct scmi_msg_base_attributes_p2a rx;
>> +
>> + ASSERT(scmi_node != NULL);
>> +
>> + INIT_LIST_HEAD(&scmi_data.channel_list);
>> + spin_lock_init(&scmi_data.channel_list_lock);
>> +
>> + if ( !acpi_disabled )
>> + {
>> + printk(XENLOG_WARNING "scmi: is not supported when using ACPI\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = scmi_check_smccc_ver();
>> + if ( ret )
>> + return ret;
>> +
>> + if ( !dt_property_read_u32(scmi_node, "arm,smc-id", &scmi_data.func_id) )
>> + {
>> + printk(XENLOG_ERR "scmi: unable to read smc-id from DT\n");
>> + return -ENOENT;
>> + }
>> +
>> + /* save shmem phandle and re-use it fro Dom0 DT shmem node */
>> + if ( !dt_property_read_u32(scmi_node, "shmem", &scmi_data.shmem_phandle) )
>> + {
>> + printk(XENLOG_ERR "scmi: unable to read shmem phandle from DT\n");
>> + return -ENOENT;
>> + }
>> +
>> + ret = scmi_dt_read_hyp_channel_addr(scmi_node, &addr, &size);
>> + if ( IS_ERR_VALUE(ret) )
>> + return -ENOENT;
>> +
>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>> + {
>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>> + return -EINVAL;
>> + }
>> +
>> + scmi_data.dt_dev = scmi_node;
>> +
>> + channel = smc_create_channel(HYP_CHANNEL, scmi_data.func_id, addr);
>> + if ( IS_ERR(channel) )
>> + goto out;
>> +
>> + ret = map_channel_memory(channel);
>> + if ( ret )
>> + goto out;
>> +
>> + channel->domain_id = DOMID_XEN;
>> +
>> + hdr.id = SCMI_BASE_PROTOCOL_ATTIBUTES;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + ret = do_smc_xfer(channel, &hdr, NULL, 0, &rx, sizeof(rx));
>> + if ( ret )
>> + goto error;
>> +
>> + n_agents = SCMI_FIELD_GET(SCMI_BASE_ATTR_NUM_AGENT, rx.attributes);
>> + printk(XENLOG_DEBUG "scmi: Got agent count %d\n", n_agents);
>> +
>> + ret = collect_agents(scmi_node);
>> + if ( ret )
>> + goto error;
>> +
>> + i = 1;
>> +
>> + list_for_each_entry(agent_channel, &scmi_data.channel_list, list)
>> + {
>> + struct scmi_msg_base_discover_agent_p2a da_rx;
>> + struct scmi_msg_base_discover_agent_a2p da_tx;
>> +
>> + ret = map_channel_memory(agent_channel);
>> + if ( ret )
>> + goto error;
>> +
>> + hdr.id = SCMI_BASE_DISCOVER_AGENT;
>> + hdr.type = 0;
>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>> +
>> + da_tx.agent_id = agent_channel->agent_id;
>> +
>> + ret = do_smc_xfer(agent_channel, &hdr, &da_tx, sizeof(da_tx), &da_rx,
>> + sizeof(da_rx));
>> + if ( agent_channel->domain_id != DOMID_XEN )
>> + unmap_channel_memory(agent_channel);
>> + if ( ret )
>> + goto error;
>> +
>> + printk(XENLOG_DEBUG "id=0x%x name=%s\n", da_rx.agent_id, da_rx.name);
>> +
>> + agent_channel->agent_id = da_rx.agent_id;
> It is OK to set agent_channel->agent_id to the value provided by the
> SCMI server, but if we are also taking the agent_channel->agent_id value
> from the user via device tree, shouldn't we throw an error if there is a
> mismatch?
>
> Or even better: can we avoid taking the value via device tree to make it
> easier to configure?
>
I plan to drop the last line, as there’s no need to overwrite agent_id.
However, we cannot avoid obtaining agent_id from the device tree because,
according to section 4.2.2.9 [0], an agent can only be discovered by its
agent_id in
a privileged domain. Alternatively, each agent can obtain its own ID by
providing
0xFFFFFFFF as agent_id and using its own channel.
[0] DEN0056E by ARM
>> + if ( i > n_agents )
>> + break;
>> +
>> + i++;
>> + }
>> +
>> + ret = sci_register(&scmi_ops);
>> + if ( ret )
>> + {
>> + printk(XENLOG_ERR "SCMI: mediator already registered (ret = %d)\n",
>> + ret);
>> + return ret;
>> + }
>> +
>> + scmi_data.initialized = true;
>> + goto out;
>> +
>> +error:
>> + unmap_channel_memory(channel);
>> + free_channel_list();
>> +out:
>> + return ret;
>> +}
>> +
>> +static const struct dt_device_match scmi_smc_match[] __initconst = {
>> + DT_MATCH_COMPATIBLE("arm,scmi-smc"),
>> + { /* sentinel */ },
>> +};
>> +
>> +DT_DEVICE_START(scmi_smc_ma, "SCMI SMC MEDIATOR", DEVICE_FIRMWARE)
>> + .dt_match = scmi_smc_match,
>> + .init = scmi_probe,
>> +DT_DEVICE_END
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
>> index 095b1a23e3..30e46de6d7 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -329,6 +329,7 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>>
>> #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
>> #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC 1
>> +#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA 2
>>
>> struct xen_arch_domainconfig {
>> /* IN/OUT */
>> @@ -355,6 +356,8 @@ struct xen_arch_domainconfig {
>> uint32_t clock_frequency;
>> /* IN */
>> uint8_t arm_sci_type;
>> + /* IN */
>> + uint8_t arm_sci_agent_id;
>> };
>> #endif /* __XEN__ || __XEN_TOOLS__ */
>>
>> --
>> 2.34.1
>>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent
2025-05-23 20:19 ` Stefano Stabellini
@ 2025-06-12 12:03 ` Oleksii Moisieiev
2025-06-18 0:35 ` Stefano Stabellini
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-12 12:03 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
On 23/05/2025 23:19, Stefano Stabellini wrote:
> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>
>> Proposal description to add separate SCMI DT node for Xen management agent
>> under "chosen" or xen-config node, like Hyperlaunch "xen,config".
> I think it is OK to place a larger "xen,config" node under /chosen with
> more information for Xen to setup SCMI more easily.
>
>
>> This proposal introduces a new approach to the Xen multi-domain
>> configuration, where all Xen-specific configuration has been moved
>> under the "/chosen" node. This requires less Dom0 device tree
>> manipulation and isolates Xen configuration from domain configuration.
>>
>> This approach provides the following device tree (DT) parameters:
>>
>> - "xen,scmi-secondary-agents": A Xen-specific parameter under the
>> "/chosen" node, which describes the SCMI agent configuration for
>> the domains.
>> - the SCMI configuration for Xen (privileged agent) and the shared
>> memory configuration for all agents are provided under the "/chosen"
>> node and are used strictly by Xen for its initial configuration.
>> - the scmi_shm and SCMI configuration for Dom0 are placed in the
>> "/firmware/scmi" node so that they can be moved to Dom0 without
>> any changes.
> Isn't the SCMI configuration present in /firmware/scmi referring to the
> privileged agent=0 meant to be used by Xen?
>
> I certainly see benefits in simplifying the configuration and especially
> reducing the amount of changes a user might have to make on the
> underlying device tree, but if the user needs to change /firmware/scmi
> with the Dom0 information, it seems more dangerous and error prone than
> the previous approach.
>
The idea is to move the privileged agent=0 configuration to the /chosen
node and
assign agent=1 to the Dom0 node under /firmware/scmi.
Benefits of This Approach:
- No Modification of the Xen DT Node Required
This eliminates the need to modify the Xen Device Tree (DT) node
before creating Dom0 in
order to set the correct shared memory (shmem).
-Consistent SCMI Configuration Format
The Dom0 DT will have the same SCMI configuration format as other
domains, simplifying the
overall configuration process.
- Unified SCMI Configuration Method
There will no longer be a need to use a different approach for SCMI
configuration in Dom0
compared to other domains.
- Separation Between Dom0 and Privileged Node
This provides a clear separation between the Dom0 node and the
privileged node.
For example:
If Dom0 only requires the clock protocol, but the Xen SCMI
configuration requires additional protocols,
this approach allows Dom0 to receive only the necessary
protocol configuration.
>> This configuration allows the use of Xen-specific nodes to provide
>> information strictly needed by Xen while using the default SCMI
>> configuration for Dom0 and other domains. As a result, no additional
>> bindings need to be introduced to the device tree.
> This is not actually implemented by this patch series, right?
It is not. Just posted this document as a proposal.
>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>> ---
>>
>>
>>
>> .../arm/firmware/arm-scmi-proposal.rst | 224 ++++++++++++++++++
>> 1 file changed, 224 insertions(+)
>> create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
>>
>> diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
>> new file mode 100644
>> index 0000000000..fcc2ed2b65
>> --- /dev/null
>> +++ b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
>> @@ -0,0 +1,224 @@
>> +
>> +Proposal for SCMI multi-agent driver bindings
>> +=============================================
>> +
>> +Now the Xen configuration for SCMI multi-agent support is done in a bit complicated way, especially
>> +from SCMI multi-agent driver initialization and Dom0 DT manipulation point of view.
>> +Also it does not take into account future requirements to support SCP SCMI FW.
>> +
>> +To enable SCMI multi-agent user need:
>> +
>> +* take host DT with basic SCMI enabled
>> +* add SCMI shared-memory nodes for all agents
>> +* update SCMI node to point on SCMI Xen management channel (``[smc-id, shmem]``)
>> +* add "xen,scmi-secondary-agents" property to the "\chosen" node
>> +
>> +.. code::
>> +
>> + chosen {
>> + xen,scmi-secondary-agents = <
>> + 1 0x82000003 &scmi_shm_1
>> + 2 0x82000004 &scmi_shm_2
>> + 3 0x82000005 &scmi_shm_3
>> + 4 0x82000006 &scmi_shm_4>;
>> + }
>> +
>> + /{
>> + // SCMI shared-memory nodes for all agents
>> + scmi_shm_0 : sram@47ff0000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
>> + };
>> + scmi_shm_1: sram@47ff1000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>> + };
>> + scmi_shm_2: sram@47ff2000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>> + };
>> + scmi_shm_3: sram@47ff3000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>> + };
>> + scmi_shm_4: sram@47ff4000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>> + };
>> +
>> + firmware {
>> + scmi: scmi {
>> + compatible = "arm,scmi-smc";
>> + arm, smc - id = <0x82000002>; <--- Xen management agent channel "smc-id"
>> + #address-cells = < 1>;
>> + #size-cells = < 0>;
>> + #access-controller-cells = < 1>;
>> + shmem = <&scmi_shm_0>; <--- Xen management agent channel "shmem"
>> +
>> + protocol@X{
>> + };
>> + };
>> + };
>> + }
>> +
>> +Important thing to note is that all information about multi-channel support is strictly Xen specific.
>> +
>> +During initialization the SCMI multi-agent driver uses Host DT SCMI node and
>> +"xen,scmi-secondary-agents" property to init itself and then, during Dom0 creation, manipulates
>> +Dom0 DT to remove Xen specific SCMI info and update dom0 SCMI nodes with Dom0 SCMI agent specific
>> +information.
>> +
>> +There are two negative points:
>> +
>> +1) Double DT modification - one is user to set up SCMI Xen support in Host DT, second -
>> + Dom0 DT manipulation.
>> +2) In case of future support of mailbox shared-memory transport there could be up to 4 mailboxes and
>> + up to 2 shared-memories per SCMI agent channel.
>> +
>> +Hence SCMI multi-agent support is Xen specific knowledge there is a proposal to add it as Xen
>> +specific DT definitions and so minimize Host and Dom0 DT manipulations.
>> +Those definitions can be added in "/chosen" or, ideally, in "xen,config" node (like in Hyperlaunch design).
>> +
>> +The SCMI binding stays generic, just two SCMI nodes defined - one for Xen management channel and
>> +one for Host Dom0 OSPM.
>> +
>> +Example of using "chosen" for configuration:
>> +
>> +.. code::
>> +
>> + /{
>> +
>> + chosen {
>> + ...
>> +
>> + // Xen SCMI management channel
>> + scmi_shm_0 : sram@47ff0000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
>> + };
>> + scmi_xen: scmi {
>> + compatible = "arm,scmi-smc";
>> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
>> + #address-cells = < 1>;
>> + #size-cells = < 0>;
>> + #access-controller-cells = < 1>;
>> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
>> + };
>> +
>> + // SCMI multi-agent configuration
>> + scmi_shm_2: sram@47ff2000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>> + };
>> + scmi_shm_3: sram@47ff3000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>> + };
>> + scmi_shm_4: sram@47ff4000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>> + };
>> + xen,scmi-secondary-agents = <
>> + 1 0x82000003 &scmi_shm
>> + 2 0x82000004 &scmi_shm_2
>> + 3 0x82000005 &scmi_shm_3
>> + 4 0x82000006 &scmi_shm_4>;
>> + };
>> +
>> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
>> + scmi_shm: sram@47ff1000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>> + };
>> +
>> + firmware {
>> + scmi: scmi {
>> + compatible = "arm,scmi-smc";
>> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
>> + #address-cells = < 1>;
>> + #size-cells = < 0>;
>> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
> By OSPM you mean Dom0 and not Xen? So this is a change compared to a
> device tree for baremetal Linux without Xen?
>
> Let me ask the same question differently. In the case of barematal Linux
> without Xen (no KVM), what would Linux see under /firmware/scmi as
> smc-id and shmem? The same as the one that Xen would use for itself? Or
> the same as the ones that Dom0 would use when Xen is present?
If this DT is used with the baremetal Linux - then the Linux Kernel will
see Dom0 "smc-id" and "shmen" under /firmware/scmi.
>> + protocol@X{
>> + };
>> + };
>> + };
>> + }
>> +
>> +
>> +In the above case:
>> +
>> +1) Xen SCMI multi-agent can be probed with DT configuration from "chosen" (or special "xen,config")
>> + node and all Xen related nodes can be easily dropped from Dom0 DT.
>> +2) Host SCMI OSPM channel DT nodes can be copied to Dom0 DT without changes if SCMI enabled for it.
>> +3) Future support for mailbox shared-memory transport (SCP SCMI FW) can be simplified as no more
>> + manipulation required with Dom0 SCMI "arm,smc-id" and "shmem" DT properties.
> Yes, I can see the benefit if we can arrange it so that the underlying
> host device tree is the same that Linux would use baremetal. And all the
> extra configuration is placed under /chosen in "xen,config" node or
> similar. I would probably call it "xen,scmi".
Personally, I would keep "xen,config" as it leaves room to add additional
configuration nodes in the future.
>> +Example of using "xen,config" for configuration:
>> +
>> +.. code::
>> +
>> + hypervisor {
>> + compatible = “hypervisor,xen”
>> +
>> + // Configuration container
>> + config {
>> + compatible = "xen,config";
>> + ...
>> +
>> + // Xen SCMI management channel
>> + scmi_shm_0 : sram@47ff0000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
>> + };
>> + scmi_xen: scmi {
>> + compatible = "arm,scmi-smc";
>> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
>> + #address-cells = < 1>;
>> + #size-cells = < 0>;
>> + #access-controller-cells = < 1>;
>> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
>> + };
>> +
>> + // SCMI multi-agent configuration
>> + scmi_shm_2: sram@47ff2000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>> + };
>> + scmi_shm_3: sram@47ff3000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>> + };
>> + scmi_shm_4: sram@47ff4000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>> + };
>> + xen,scmi-secondary-agents = <
>> + 1 0x82000003 &scmi_shm
>> + 2 0x82000004 &scmi_shm_2
>> + 3 0x82000005 &scmi_shm_3
>> + 4 0x82000006 &scmi_shm_4>;
>> + };
>> + };
>> +
>> + /{
>> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
>> + scmi_shm: sram@47ff1000 {
>> + compatible = "arm,scmi-shmem";
>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>> + };
>> +
>> + firmware {
>> + scmi: scmi {
>> + compatible = "arm,scmi-smc";
>> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
>> + #address-cells = < 1>;
>> + #size-cells = < 0>;
>> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
>> +
>> + protocol@X{
>> + };
>> + };
>> + };
>> + }
>> --
>> 2.34.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-02 7:17 ` Bertrand Marquis
@ 2025-06-12 12:03 ` Oleksii Moisieiev
2025-06-12 12:10 ` Grygorii Strashko
1 sibling, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-12 12:03 UTC (permalink / raw)
To: Bertrand Marquis, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Jan Beulich, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On 02/06/2025 10:17, Bertrand Marquis wrote:
> Hi Stefano and Oleksii,
>
>> On 23 May 2025, at 22:06, Stefano Stabellini<sstabellini@kernel.org> wrote:
>>
>> One question for Bertrand below
>>
>>
>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
>>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
>>> below.
>>>
>>> +-----------------------------------------+
>>> | |
>>> | EL3 TF-A SCMI |
>>> +-------+--+-------+--+-------+--+-------++
>>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
>>> +-----+-+ +---+---+ +--+----+ +---+---+
>>> smc-id0 | | | |
>>> agent0 | | | |
>>> +-----v--------+---------+-----------+----+
>>> | | | | |
>>> | | | | |
>>> +--------------+---------+-----------+----+
>>> smc-id1 | smc-id2| smc-idX|
>>> agent1 | agent2 | agentX |
>>> | | |
>>> +----v---+ +--v-----+ +--v-----+
>>> | | | | | |
>>> | Dom0 | | Dom1 | | DomX |
>>> | | | | | |
>>> | | | | | |
>>> +--------+ +--------+ +--------+
>>>
>>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
>>> memory transport for every Agent in the system.
>>>
>>> The SCMI Agent transport channel defined by pair:
>>> - smc-id: SMC/HVC id used for Doorbell
>>> - shmem: shared memory for messages transfer, Xen page aligned,
>>> p2m_mmio_direct_nc.
>>>
>>> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
>>> multi-agent functionality under Xen:
>>> - Xen manegement agent: trusted agents that accesses to the Base Protocol
>>> commands to configure agent specific permissions
>>> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
>>> allowed direct HW access. At least one OSPM VM agent has to be provided
>>> by FW if HW is handled only by Dom0 or Driver Domain.
>>>
>>> The EL3 SCMI FW expected to implement following Base protocol messages:
>>> - BASE_DISCOVER_AGENT
>>> - BASE_RESET_AGENT_CONFIGURATION (optional)
>>> - BASE_SET_DEVICE_PERMISSIONS (optional)
>>>
>>> The SCI SCMI SMC multi-agent driver implements following functionality:
>>> - It's initialized based on the Host DT SCMI node (only one SCMI interface
>>> is supported) which describes Xen management agent SCMI interface.
>>>
>>> scmi_shm_0 : sram@47ff0000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff0000 0x0 0x1000>;
>>> };
>>> firmware {
>>> scmi: scmi {
>>> compatible = "arm,scmi-smc";
>>> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
>> some extra spaces, it might be a copy/paste error
>>
>>
>>> \#address-cells = < 1>;
>>> \#size-cells = < 0>;
>>> \#access-controller - cells = < 1>;
>>> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
>>>
>>> protocol@X{
>>> };
>>> };
>>> };
>>>
>>> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
>>> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
>>>
>>> chosen {
>>> xen,scmi-secondary-agents = <
>>> 1 0x82000003 &scmi_shm_1
>>> 2 0x82000004 &scmi_shm_2
>>> 3 0x82000005 &scmi_shm_3
>>> 4 0x82000006 &scmi_shm_4>;
>>> }
>>>
>>> /{
>>> scmi_shm_1: sram@47ff1000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff1000 0x0 0x1000>;
>>> };
>>> scmi_shm_2: sram@47ff2000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff2000 0x0 0x1000>;
>>> };
>>> scmi_shm_3: sram@47ff3000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff3000 0x0 0x1000>;
>>> };
>>> }
>>> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
>>> this agent_id.
>>>
>>> Note that Xen is the only one entry in the system which need to know
>>> about SCMI multi-agent support.
>>>
>>> - It implements the SCI subsystem interface required for configuring and
>>> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
>>> SCMI functionality for domain it has to be configured with unique supported
>>> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
>>> [smc-id, shmem] defined for this SCMI Agent_id.
>>> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
>>> -- zero-copy, the guest domain puts SCMI message in shmem;
>>> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
>>> -- the Xen driver catches exception, do checks and synchronously forwards
>>> it to EL3 FW.
>>> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
>>> management agent channel on domain destroy event. This allows to reset
>>> resources used by domain and so implement use-case like domain reboot.
>>>
>>> Dom0 Enable SCMI SMC:
>>> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
>>> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
>>> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
>>> node according to assigned agent_id.
>>>
>>> Guest domains enable SCMI SMC:
>>> - xl.cfg: add configuration option as below
>>>
>>> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
>>>
>>> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
>>> the domain, for example:
>>>
>>> iomem = [
>>> "47ff2,1@22001",
>>> ]
>> Looking at the code and the configuration options, it looks like it is
>> possible to map a scmi-shmem channel at a different address for the
>> guest. It seems like it would work. Is that correct?
>>
>>
>>> - DT: add SCMI nodes to the Driver domain partial device tree as in the
>>> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
>>>
>>> passthrough {
>>> scmi_shm_0: sram@22001000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x22001000 0x0 0x1000>;
>>> };
>>>
>>> firmware {
>>> compatible = "simple-bus";
>>> scmi: scmi {
>>> compatible = "arm,scmi-smc";
>>> arm,smc-id = <0x82000004>;
>>> shmem = <&scmi_shm_0>;
>>> ...
>>> }
>>> }
>>> }
>>>
>>> SCMI "4.2.1.1 Device specific access control"
>>>
>>> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
>>> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
>>> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
>>> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
>>> to the Xen SCMI.
>>>
>>> &i2c1 {
>>> access-controllers = <&scmi 0>;
>>> };
>>>
>>> The Dom0 and dom0less domains DT devices will be processed automatically through
>>> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
>>> shell be used:
>>>
>>> dtdev = [
>>> "/soc/i2c@e6508000",
>>> ]
>>>
>>> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
>>>
>>> [1]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
>>> [2]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>> Thanks for the long explanation, great work! I am really looking forward
>> to have this feature in the tree soon.
>>
>>
>>> ---
>>>
>>> Changes in v4:
>>> - toolstack comments from Anthony PERARD
>>> - added dom0less support
>>> - added doc for "xen,scmi-secondary-agents"
>>>
>>> docs/man/xl.cfg.5.pod.in | 13 +
>>> docs/misc/arm/device-tree/booting.txt | 60 ++
>>> docs/misc/xen-command-line.pandoc | 9 +
>>> tools/libs/light/libxl_arm.c | 4 +
>>> tools/libs/light/libxl_types.idl | 4 +-
>>> tools/xl/xl_parse.c | 12 +
>>> xen/arch/arm/dom0less-build.c | 11 +
>>> xen/arch/arm/domain_build.c | 3 +-
>>> xen/arch/arm/firmware/Kconfig | 11 +
>>> xen/arch/arm/firmware/Makefile | 1 +
>>> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
>>> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
>>> xen/arch/arm/firmware/scmi-shmem.h | 45 +
>>> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
>>> xen/include/public/arch-arm.h | 3 +
>>> 15 files changed, 1371 insertions(+), 2 deletions(-)
>>> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
>>> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
>>>
>>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>>> index 1ccf50b8ea..302c46d8bc 100644
>>> --- a/docs/man/xl.cfg.5.pod.in
>>> +++ b/docs/man/xl.cfg.5.pod.in
>>> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
>>> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
>>> option.
>>>
>>> +=item B<scmi_smc_multiagent>
>>> +
>>> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>>> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
>>> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
>>> +specified for the guest.
>>> +
>>> =back
>>>
>>> +=item B<agent_id=NUMBER>
>>> +
>>> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
>>> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
>>> +existing on a single host must be unique and in the range [1..255].
>>> +
>>> =back
>>>
>>> =back
>>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
>>> index 8943c04173..c8923ab8b2 100644
>>> --- a/docs/misc/arm/device-tree/booting.txt
>>> +++ b/docs/misc/arm/device-tree/booting.txt
>>> @@ -296,6 +296,20 @@ with the following properties:
>>> Should be used together with dom0_scmi_smc_passthrough Xen command line
>>> option.
>>>
>>> + - "scmi_smc_multiagent"
>>> +
>>> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>>> + SMC calls forwarding from domain to the EL3 firmware (like ARM
>>> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
>>> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
>>> + property.
>>> +
>>> +- "xen,sci_agent_id"
>>> +
>>> + Specifies a non-zero ARM SCI agent id for the guest. This option is
>>> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
>>> + the guest. The agent ids of guest must be unique and in the range [1..255].
>>> +
>>> Under the "xen,domain" compatible node, one or more sub-nodes are present for the DomU
>>> kernel and ramdisk. @@ -764,3 +778,49 @@ The automatically allocated
>>> static shared memory will get mapped at 0x80000000 in DomU1 guest
>>> physical address space, and at 0x90000000 in DomU2 guest physical
>>> address space. DomU1 is explicitly defined as the owner domain, and
>>> DomU2 is the borrower domain. + +SCMI SMC multi-agent support
>>> +============================ + +For enabling the ARM SCMI SMC
>>> multi-agent support (enabled by CONFIG_SCMI_SMC_MA) +the Xen
>>> specific SCMI Agent's configuration shell be provided in the Host DT
>>> +according to the SCMI compliant EL3 Firmware specification with
>>> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
>>> +the top-level "chosen" node:
>>> +
>>> +- xen,scmi-secondary-agents
>>> +
>>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
>>> + available for Xen. Each Agent defined as triple consisting of:
>>> + SCMI agent_id,
>>> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
>>> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
>>> +
>>> +As an example:
>>> +
>>> +chosen {
>>> + xen,scmi-secondary-agents = <
>>> + 1 0x82000003 &scmi_shm_1
>>> + 2 0x82000004 &scmi_shm_2
>>> + 3 0x82000005 &scmi_shm_3
>>> + 4 0x82000006 &scmi_shm_4>;
>>> +}
>> NIT: it should be };
>>
>> Looking at scmi_probe, collect_agents, and the following SCMI
>> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
>> information?
>>
>> It looks like we can discover the agend_ids for every channel, I guess
>> what we need to know is the shmem location for every channel? But the
>> full list of shmem channel is available below thanks to the scmi-shmem
>> nodes.
>>
>> So, we have the list of scmi-shmem anyway, and we can probe the
>> agent_id. The only parameter left is the smc_id/func_id.
>>
>> Or maybe smc_id/func_id can be calculated from agent_id?
>>
>> I am asking mostly because if a user is supposed to add this
>> xen,scmi-secondary-agents property, where are they supposed to find the
>> smc_id/func_id information?
>>
>> It is important that we write down in this document how the user is
>> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
>>
>>
>>> +/{
>>> + scmi_shm_1: sram@47ff1000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>>> + };
>>> + scmi_shm_2: sram@47ff2000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>>> + };
>>> + scmi_shm_3: sram@47ff3000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>>> + };
>>> + scmi_shm_3: sram@47ff4000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>>> + };
>> Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
>> Or are under /firmware? Or are they under /chosen?
>>
>> I take they are under the top level node together with scmi_shm_0?
>>
>> Can you please also clarify in the document as well?
>>
>>
>>> +}
>>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
>>> index 8e50f6b7c7..bc3c64d6ec 100644
>>> --- a/docs/misc/xen-command-line.pandoc
>>> +++ b/docs/misc/xen-command-line.pandoc
>>> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
>>> SCMI nodes removed from Dom0/hwdom device tree.
>>> (for example, thin Dom0 with Driver domain use-case).
>>>
>>> +### dom0_scmi_agent_id (ARM)
>>> +> `= <integer>`
>>> +
>>> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
>>> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
>>> +The SCMI will be disabled for Dom0 if this option is not specified
>>> +(for example, thin Dom0 or dom0less use-cases).
>>> +The agent ids of domains existing on a single host must be unique.
>>> +
>>> ### dtuart (ARM)
>>>> `= path [:options]`
>>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>>> index 28ba9eb787..7712f53cd4 100644
>>> --- a/tools/libs/light/libxl_arm.c
>>> +++ b/tools/libs/light/libxl_arm.c
>>> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
>>> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>> break;
>>> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
>>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
>>> + break;
>>> default:
>>> LOG(ERROR, "Unknown ARM_SCI type %d",
>>> d_config->b_info.arch_arm.arm_sci.type);
>>> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
>>> index aa2190ab5b..11e31ce786 100644
>>> --- a/tools/libs/light/libxl_types.idl
>>> +++ b/tools/libs/light/libxl_types.idl
>>> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
>>>
>>> libxl_arm_sci_type = Enumeration("arm_sci_type", [
>>> (0, "none"),
>>> - (1, "scmi_smc")
>>> + (1, "scmi_smc"),
>>> + (2, "scmi_smc_multiagent")
>>> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
>>>
>>> libxl_arm_sci = Struct("arm_sci", [
>>> ("type", libxl_arm_sci_type),
>>> + ("agent_id", uint8)
>>> ])
>>>
>>> libxl_rdm_reserve = Struct("rdm_reserve", [
>>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>>> index bd22be9d33..81aa3797e3 100644
>>> --- a/tools/xl/xl_parse.c
>>> +++ b/tools/xl/xl_parse.c
>>> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
>>> }
>>> }
>>>
>>> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
>>> + unsigned long val = parse_ulong(oparg);
>>> +
>>> + if (!val || val > 255) {
>>> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
>>> + val);
>>> + ret = ERROR_INVAL;
>>> + goto parse_error;
>>> + }
>>> + arm_sci->agent_id = val;
>>> + }
>>> +
>>> ptr = strtok(NULL, ",");
>>> }
>>>
>>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>>> index 0a00f03a25..43d21eb889 100644
>>> --- a/xen/arch/arm/dom0less-build.c
>>> +++ b/xen/arch/arm/dom0less-build.c
>>> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>>> else if ( !strcmp(sci_type, "scmi_smc") )
>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
>>> + {
>>> + uint32_t agent_id = 0;
>>> +
>>> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
>>> + !agent_id )
>> shouldn't we check that agent_id <= 255 ?
>>
>>
>>> + return -EINVAL;
>>> +
>>> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>> + d_cfg->arch.arm_sci_agent_id = agent_id;
>>> + }
>>> else
>>> {
>>> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index 36d28b52a4..0c9274a2b3 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
>>> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
>>> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
>>> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
>>> continue;
>>>
>>> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") ) diff --git a/xen/arch/arm/firmware/Kconfig
>>> b/xen/arch/arm/firmware/Kconfig index 5c5f0880c4..6b051c8ada 100644
>>> --- a/xen/arch/arm/firmware/Kconfig +++
>>> b/xen/arch/arm/firmware/Kconfig @@ -29,6 +29,17 @@ config SCMI_SMC
>>> driver domain. Use with EL3 firmware which supports only single SCMI
>>> OSPM agent. +config SCMI_SMC_MA + bool "Enable ARM SCMI SMC multi-agent driver"
>>> + select ARM_SCI
>>> + help
>>> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
>>> + to EL3 firmware (TF-A) which supports multi-agent feature.
>>> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
>>> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
>>> + allowed platform resources through dedicated SMC/HVC Shared memory based
>>> + transport.
>>> +
>>> endchoice
>>>
>>> endmenu
>>> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
>>> index 71bdefc24a..37927e690e 100644
>>> --- a/xen/arch/arm/firmware/Makefile
>>> +++ b/xen/arch/arm/firmware/Makefile
>>> @@ -1,2 +1,3 @@
>>> obj-$(CONFIG_ARM_SCI) += sci.o
>>> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
>>> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
>>> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
>>> new file mode 100644
>>> index 0000000000..3f4b9c5d6b
>>> --- /dev/null
>>> +++ b/xen/arch/arm/firmware/scmi-proto.h
>>> @@ -0,0 +1,164 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * Arm System Control and Management Interface definitions
>>> + * Version 3.0 (DEN0056C)
>>> + *
>>> + * Copyright (c) 2024 EPAM Systems
>>> + */
>>> +
>>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>>> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>> NIT: ARM_FIRMWARE_SCMI_PROTO_H
>>
>>
>>> +#include <xen/stdint.h>
>>> +
>>> +#define SCMI_SHORT_NAME_MAX_SIZE 16
>>> +
>>> +/* SCMI status codes. See section 4.1.4 */
>>> +#define SCMI_SUCCESS 0
>>> +#define SCMI_NOT_SUPPORTED (-1)
>>> +#define SCMI_INVALID_PARAMETERS (-2)
>>> +#define SCMI_DENIED (-3)
>>> +#define SCMI_NOT_FOUND (-4)
>>> +#define SCMI_OUT_OF_RANGE (-5)
>>> +#define SCMI_BUSY (-6)
>>> +#define SCMI_COMMS_ERROR (-7)
>>> +#define SCMI_GENERIC_ERROR (-8)
>>> +#define SCMI_HARDWARE_ERROR (-9)
>>> +#define SCMI_PROTOCOL_ERROR (-10)
>>> +
>>> +/* Protocol IDs */
>>> +#define SCMI_BASE_PROTOCOL 0x10
>>> +
>>> +/* Base protocol message IDs */
>>> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
>>> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
>>> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
>>> +#define SCMI_BASE_DISCOVER_AGENT 0x7
>>> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
>>> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
>>> +
>>> +typedef struct scmi_msg_header {
>>> + uint8_t id;
>>> + uint8_t type;
>>> + uint8_t protocol;
>>> + uint32_t status;
>>> +} scmi_msg_header_t;
>>> +
>>> +/* Table 2 Message header format */
>>> +#define SCMI_HDR_ID GENMASK(7, 0)
>>> +#define SCMI_HDR_TYPE GENMASK(9, 8)
>>> +#define SCMI_HDR_PROTO GENMASK(17, 10)
>>> +
>>> +#define SCMI_FIELD_GET(_mask, _reg) \
>>> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
>>> +#define SCMI_FIELD_PREP(_mask, _val) \
>>> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
>>> +
>>> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
>>> +{
>>> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
>>> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
>>> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
>>> +}
>>> +
>>> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
>>> +{
>>> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
>>> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
>>> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
>>> +}
>>> +
>>> +static inline int scmi_to_xen_errno(int scmi_status)
>>> +{
>>> + if ( scmi_status == SCMI_SUCCESS )
>>> + return 0;
>>> +
>>> + switch ( scmi_status )
>>> + {
>>> + case SCMI_NOT_SUPPORTED:
>>> + return -EOPNOTSUPP;
>>> + case SCMI_INVALID_PARAMETERS:
>>> + return -EINVAL;
>>> + case SCMI_DENIED:
>>> + return -EACCES;
>>> + case SCMI_NOT_FOUND:
>>> + return -ENOENT;
>>> + case SCMI_OUT_OF_RANGE:
>>> + return -ERANGE;
>>> + case SCMI_BUSY:
>>> + return -EBUSY;
>>> + case SCMI_COMMS_ERROR:
>>> + return -ENOTCONN;
>>> + case SCMI_GENERIC_ERROR:
>>> + return -EIO;
>>> + case SCMI_HARDWARE_ERROR:
>>> + return -ENXIO;
>>> + case SCMI_PROTOCOL_ERROR:
>>> + return -EBADMSG;
>>> + default:
>>> + return -EINVAL;
>>> + }
>>> +}
>>> +
>>> +/* PROTOCOL_VERSION */
>>> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
>>> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
>>> +
>>> +struct scmi_msg_prot_version_p2a {
>>> + uint32_t version;
>>> +} __packed;
>>> +
>>> +/* BASE PROTOCOL_ATTRIBUTES */
>>> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
>>> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
>>> +
>>> +struct scmi_msg_base_attributes_p2a {
>>> + uint32_t attributes;
>>> +} __packed;
>>> +
>>> +/*
>>> + * BASE_DISCOVER_AGENT
>>> + */
>>> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
>>> +
>>> +struct scmi_msg_base_discover_agent_a2p {
>>> + uint32_t agent_id;
>>> +} __packed;
>>> +
>>> +struct scmi_msg_base_discover_agent_p2a {
>>> + uint32_t agent_id;
>>> + char name[SCMI_SHORT_NAME_MAX_SIZE];
>>> +} __packed;
>>> +
>>> +/*
>>> + * BASE_SET_DEVICE_PERMISSIONS
>>> + */
>>> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
>>> +
>>> +struct scmi_msg_base_set_device_permissions_a2p {
>>> + uint32_t agent_id;
>>> + uint32_t device_id;
>>> + uint32_t flags;
>>> +} __packed;
>>> +
>>> +/*
>>> + * BASE_RESET_AGENT_CONFIGURATION
>>> + */
>>> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
>>> +
>>> +struct scmi_msg_base_reset_agent_cfg_a2p {
>>> + uint32_t agent_id;
>>> + uint32_t flags;
>>> +} __packed;
>>> +
>>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
>>> +
>>> +/*
>>> + * Local variables:
>>> + * mode: C
>>> + * c-file-style: "BSD"
>>> + * c-basic-offset: 4
>>> + * tab-width: 4
>>> + * indent-tabs-mode: nil
>>> + * End:
>>> + */
>>> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
>>> new file mode 100644
>>> index 0000000000..dd613ee0b5
>>> --- /dev/null
>>> +++ b/xen/arch/arm/firmware/scmi-shmem.c
>>> @@ -0,0 +1,173 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>>> + *
>>> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>> + * Copyright (c) 2025 EPAM Systems
>>> + */
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +
>>> +#include <asm/io.h>
>>> +#include <xen/err.h>
>>> +
>>> +#include "scmi-proto.h"
>>> +#include "scmi-shmem.h"
>> This code is written more generically than the description implies. If
>> we only want to make SMC calls to TF-A on EL3 and exchange data with it
>> over shared memory, then I think:
>> - we don't need the __iomem tag, as there is no MMIO
> agree
>
>> - we only need a DMB, not a DSB (readl and writel imply DSB, use only
>> readl_relaxed and writel_relaxed)
> agree
>
>> On the other hand, if we also want to handle the case where the SCMI
>> server could be on a separate co-processor, then what this code is doing
>> is not sufficient because we also need a dcache flush, in addition to
>> the DSB.
>>
>> Bertrand, can you double-check?
> If we want to handle a case where the memory is accessible to a coprocessor
> but there is no cache coherency, we need to flush the dcache definitely.
>
> Seeing the amount of data here, I do agree with Stefano that it would be a good
> idea to make the provision to flush the data cache in all cases. Even if the data
> is accessed by a secure partition or the firmware coherently, flushing in all cases
> would have very limited performance impact here.
>
> There is the other solution to have some kind of parameter to say if the accessor
> has coherent cache access but I do not think the performance impact here would
> justify such a complexity.
>
> Cheers
> Bertrand
>
Hmm, I think we need more input regarding memory access since ARM uses
memcpy_{from/to}io in the Linux kernel driver.
It might be worth involving Cristian Marussi in the discussion, as he is
the author of the Linux driver. What do you think?
>>> +/*
>>> TODO remove: fixed+ * Copy data from IO memory space to "real" memory space.
>>> + */
>>> +static void __memcpy_fromio(void *to, const volatile void __iomem *from,
>>> + size_t count)
>>> +{
>>> + while ( count && !IS_ALIGNED((unsigned long)from, 4) )
>>> + {
>>> + *(u8 *)to = readb_relaxed(from);
>>> + from++;
>>> + to++;
>>> + count--;
>>> + }
>>> +
>>> + while ( count >= 4 )
>>> + {
>>> + *(u32 *)to = readl_relaxed(from);
>>> + from += 4;
>>> + to += 4;
>>> + count -= 4;
>>> + }
>>> +
>>> + while ( count )
>>> + {
>>> + *(u8 *)to = readb_relaxed(from);
>>> + from++;
>>> + to++;
>>> + count--;
>>> + }
>>> +}
>>> +
>>> +/*
>>> + * Copy data from "real" memory space to IO memory space.
>>> + */
>>> +static void __memcpy_toio(volatile void __iomem *to, const void *from,
>>> + size_t count)
>>> +{
>>> + while ( count && !IS_ALIGNED((unsigned long)to, 4) )
>>> + {
>>> + writeb_relaxed(*(u8 *)from, to);
>>> + from++;
>>> + to++;
>>> + count--;
>>> + }
>>> +
>>> + while ( count >= 4 )
>>> + {
>>> + writel_relaxed(*(u32 *)from, to);
>>> + from += 4;
>>> + to += 4;
>>> + count -= 4;
>>> + }
>>> +
>>> + while ( count )
>>> + {
>>> + writeb_relaxed(*(u8 *)from, to);
>>> + from++;
>>> + to++;
>>> + count--;
>>> + }
>>> +}
>> I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
>> we use a simple memcpy?
>>
>>
>>> +static inline int
>>> +shmem_channel_is_free(const volatile struct scmi_shared_mem __iomem *shmem)
>>> +{
>>> + return (readl(&shmem->channel_status) &
>>> + SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE) ? 0 : -EBUSY;
>>> +}
>>> +
>>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>>> + scmi_msg_header_t *hdr, void *data, int len)
>>> +{
>>> + int ret;
>>> +
>>> + if ( (len + sizeof(shmem->msg_header)) > SCMI_SHMEM_MAPPED_SIZE )
>>> + {
>>> + printk(XENLOG_ERR "scmi: Wrong size of smc message. Data is invalid\n");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + ret = shmem_channel_is_free(shmem);
>>> + if ( ret )
>>> + return ret;
>>> +
>>> + writel_relaxed(0x0, &shmem->channel_status);
>>> + /* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
>>> + writel_relaxed(0x0, &shmem->flags);
>>> + writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
>>> + writel(pack_scmi_header(hdr), &shmem->msg_header);
>>> +
>>> + if ( len > 0 && data )
>>> + __memcpy_toio(shmem->msg_payload, data, len);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>>> + scmi_msg_header_t *hdr, void *data, int len)
>>> +{
>>> + int recv_len;
>>> + int ret;
>>> + int pad = sizeof(hdr->status);
>>> +
>>> + if ( len >= SCMI_SHMEM_MAPPED_SIZE - sizeof(shmem) )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: Wrong size of input smc message. Data may be invalid\n");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + ret = shmem_channel_is_free(shmem);
>>> + if ( ret )
>>> + return ret;
>>> +
>>> + recv_len = readl(&shmem->length) - sizeof(shmem->msg_header);
>>> +
>>> + if ( recv_len < 0 )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: Wrong size of smc message. Data may be invalid\n");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + unpack_scmi_header(readl(&shmem->msg_header), hdr);
>>> +
>>> + hdr->status = readl(&shmem->msg_payload);
>>> + recv_len = recv_len > pad ? recv_len - pad : 0;
>>> +
>>> + ret = scmi_to_xen_errno(hdr->status);
>>> + if ( ret )
>>> + {
>>> + printk(XENLOG_DEBUG "scmi: Error received: %d\n", ret);
>>> + return ret;
>>> + }
>>> +
>>> + if ( recv_len > len )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: Not enough buffer for message %d, expecting %d\n",
>>> + recv_len, len);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if ( recv_len > 0 )
>>> + __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +/*
>>> + * Local variables:
>>> + * mode: C
>>> + * c-file-style: "BSD"
>>> + * c-basic-offset: 4
>>> + * tab-width: 4
>>> + * indent-tabs-mode: nil
>>> + * End:
>>> + */
>>> diff --git a/xen/arch/arm/firmware/scmi-shmem.h b/xen/arch/arm/firmware/scmi-shmem.h
>>> new file mode 100644
>>> index 0000000000..2f8e23ff76
>>> --- /dev/null
>>> +++ b/xen/arch/arm/firmware/scmi-shmem.h
>>> @@ -0,0 +1,45 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * Arm System Control and Management Interface definitions
>>> + * Version 3.0 (DEN0056C)
>>> + * Shared Memory based Transport
>>> + *
>>> + * Copyright (c) 2024 EPAM Systems
>>> + */
>>> +
>>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
>>> +#define XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
>> NIT: ARM_FIRMWARE_SCMI_SHMEM_H
>>
>>
>>> +#include <xen/stdint.h>
>>> +
>>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE BIT(0, UL)
>>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_ERROR BIT(1, UL)
>>> +
>>> +struct scmi_shared_mem {
>>> + uint32_t reserved;
>>> + uint32_t channel_status;
>>> + uint32_t reserved1[2];
>>> + uint32_t flags;
>>> + uint32_t length;
>>> + uint32_t msg_header;
>>> + uint8_t msg_payload[];
>>> +};
>>> +
>>> +#define SCMI_SHMEM_MAPPED_SIZE PAGE_SIZE
>>> +
>>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>>> + scmi_msg_header_t *hdr, void *data, int len);
>>> +
>>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>>> + scmi_msg_header_t *hdr, void *data, int len);
>>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_ */
>>> +
>>> +/*
>>> + * Local variables:
>>> + * mode: C
>>> + * c-file-style: "BSD"
>>> + * c-basic-offset: 4
>>> + * tab-width: 4
>>> + * indent-tabs-mode: nil
>>> + * End:
>>> + */
>>> diff --git a/xen/arch/arm/firmware/scmi-smc-multiagent.c b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>>> new file mode 100644
>>> index 0000000000..e023bca3a1
>>> --- /dev/null
>>> +++ b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>>> @@ -0,0 +1,860 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>>> + *
>>> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>> + * Copyright (c) 2025 EPAM Systems
>>> + */
>>> +
>>> +#include <xen/acpi.h>
>>> +
>>> +#include <xen/device_tree.h>
>>> +#include <xen/init.h>
>>> +#include <xen/iocap.h>
>>> +#include <xen/err.h>
>>> +#include <xen/libfdt/libfdt.h>
>>> +#include <xen/param.h>
>>> +#include <xen/sched.h>
>>> +#include <xen/vmap.h>
>>> +
>>> +#include <asm/firmware/sci.h>
>>> +#include <asm/smccc.h>
>>> +
>>> +#include "scmi-proto.h"
>>> +#include "scmi-shmem.h"
>>> +
>>> +#define SCMI_AGENT_ID_INVALID 0xFF
>>> +
>>> +static uint8_t __initdata opt_dom0_scmi_agent_id = SCMI_AGENT_ID_INVALID;
>>> +integer_param("dom0_scmi_agent_id", opt_dom0_scmi_agent_id);
>>> +
>>> +#define SCMI_SECONDARY_AGENTS "xen,scmi-secondary-agents"
>>> +
>>> +#define HYP_CHANNEL 0x0
>>> +
>>> +struct scmi_channel {
>>> + uint32_t agent_id;
>>> + uint32_t func_id;
>>> + domid_t domain_id;
>>> + uint64_t paddr;
>>> + uint64_t len;
>>> + struct scmi_shared_mem __iomem *shmem;
>>> + spinlock_t lock;
>>> + struct list_head list;
>>> +};
>>> +
>>> +struct scmi_data {
>>> + struct list_head channel_list;
>>> + spinlock_t channel_list_lock;
>>> + uint32_t func_id;
>>> + bool initialized;
>>> + uint32_t shmem_phandle;
>>> + struct dt_device_node *dt_dev;
>>> +};
>>> +
>>> +static struct scmi_data scmi_data;
>>> +
>>> +static int send_smc_message(struct scmi_channel *chan_info,
>>> + scmi_msg_header_t *hdr, void *data, int len)
>>> +{
>>> + struct arm_smccc_res resp;
>>> + int ret;
>>> +
>>> + ret = shmem_put_message(chan_info->shmem, hdr, data, len);
>>> + if ( ret )
>>> + return ret;
>>> +
>>> + arm_smccc_1_1_smc(chan_info->func_id, 0, 0, 0, 0, 0, 0, 0, &resp);
>>> +
>>> + if ( resp.a0 )
>>> + return -EOPNOTSUPP;
>> Why if repo.a0 != 0 then we assume -EOPNOTSUPP? Is this part of the SCMI
>> specification?
>>
>>
>>> + return 0;
>>> +}
>>> +
>>> +static int do_smc_xfer(struct scmi_channel *chan_info, scmi_msg_header_t *hdr,
>>> + void *tx_data, int tx_size, void *rx_data, int rx_size)
>>> +{
>>> + int ret = 0;
>>> +
>>> + ASSERT(chan_info && chan_info->shmem);
>>> +
>>> + if ( !hdr )
>>> + return -EINVAL;
>>> +
>>> + spin_lock(&chan_info->lock);
>>> +
>>> + printk(XENLOG_DEBUG
>>> + "scmi: agent_id = %d msg_id = %x type = %d, proto = %x\n",
>>> + chan_info->agent_id, hdr->id, hdr->type, hdr->protocol);
>>> +
>>> + ret = send_smc_message(chan_info, hdr, tx_data, tx_size);
>>> + if ( ret )
>>> + goto clean;
>>> +
>>> + ret = shmem_get_response(chan_info->shmem, hdr, rx_data, rx_size);
>>> +
>>> +clean:
>>> + printk(XENLOG_DEBUG
>>> + "scmi: get smc response agent_id = %d msg_id = %x proto = %x res=%d\n",
>>> + chan_info->agent_id, hdr->id, hdr->protocol, ret);
>>> +
>>> + spin_unlock(&chan_info->lock);
>>> +
>>> + return ret;
>>> +}
>>> +
>>> +static struct scmi_channel *get_channel_by_id(uint32_t agent_id)
>>> +{
>>> + struct scmi_channel *curr;
>>> + bool found = false;
>>> +
>>> + spin_lock(&scmi_data.channel_list_lock);
>>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>>> + {
>>> + if ( curr->agent_id == agent_id )
>>> + {
>>> + found = true;
>>> + break;
>>> + }
>>> + }
>>> +
>>> + spin_unlock(&scmi_data.channel_list_lock);
>>> + if ( found )
>>> + return curr;
>>> +
>>> + return NULL;
>>> +}
>>> +
>>> +static struct scmi_channel *acquire_scmi_channel(struct domain *d,
>>> + uint32_t agent_id)
>>> +{
>>> + struct scmi_channel *curr;
>>> + struct scmi_channel *ret = ERR_PTR(-ENOENT);
>>> +
>>> + spin_lock(&scmi_data.channel_list_lock);
>>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>>> + {
>>> + if ( curr->agent_id == agent_id )
>>> + {
>>> + if ( curr->domain_id != DOMID_INVALID )
>>> + {
>>> + ret = ERR_PTR(-EEXIST);
>>> + break;
>>> + }
>>> +
>>> + curr->domain_id = d->domain_id;
>>> + ret = curr;
>>> + break;
>>> + }
>>> + }
>>> +
>>> + spin_unlock(&scmi_data.channel_list_lock);
>>> +
>>> + return ret;
>>> +}
>>> +
>>> +static void relinquish_scmi_channel(struct scmi_channel *channel)
>>> +{
>>> + ASSERT(channel != NULL);
>>> +
>>> + spin_lock(&scmi_data.channel_list_lock);
>>> + channel->domain_id = DOMID_INVALID;
>>> + spin_unlock(&scmi_data.channel_list_lock);
>>> +}
>>> +
>>> +static int map_channel_memory(struct scmi_channel *channel)
>>> +{
>>> + ASSERT(channel && channel->paddr);
>>> + channel->shmem = ioremap_nocache(channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
>> ioremap is for MMIO, if these shared memory channels are on DDR, then it
>> would not be the right call. Are the "arm,scmi-shmem" address ranges
>> part of the memory node ranges? Or are they completely separate?
>>
>> Also, why nocache? Wouldn't we want ioremap_cache?
>>
>>
>>> + if ( !channel->shmem )
>>> + return -ENOMEM;
>>> +
>>> + channel->shmem->channel_status = SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE;
>>> + printk(XENLOG_DEBUG "scmi: Got shmem %lx after vmap %p\n", channel->paddr,
>>> + channel->shmem);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static void unmap_channel_memory(struct scmi_channel *channel)
>>> +{
>>> + ASSERT(channel && channel->shmem);
>>> + iounmap(channel->shmem);
>>> + channel->shmem = NULL;
>>> +}
>>> +
>>> +static struct scmi_channel *smc_create_channel(uint32_t agent_id,
>>> + uint32_t func_id, uint64_t addr)
>>> +{
>>> + struct scmi_channel *channel;
>>> +
>>> + channel = get_channel_by_id(agent_id);
>>> + if ( channel )
>>> + return ERR_PTR(EEXIST);
>>> +
>>> + channel = xmalloc(struct scmi_channel);
>>> + if ( !channel )
>>> + return ERR_PTR(ENOMEM);
>>> +
>>> + spin_lock_init(&channel->lock);
>>> + channel->agent_id = agent_id;
>>> + channel->func_id = func_id;
>>> + channel->domain_id = DOMID_INVALID;
>>> + channel->shmem = NULL;
>>> + channel->paddr = addr;
>>> + list_add_tail(&channel->list, &scmi_data.channel_list);
>>> + return channel;
>>> +}
>>> +
>>> +static void free_channel_list(void)
>>> +{
>>> + struct scmi_channel *curr, *_curr;
>>> +
>>> + list_for_each_entry_safe(curr, _curr, &scmi_data.channel_list, list)
>>> + {
>>> + list_del(&curr->list);
>>> + xfree(curr);
>>> + }
>>> +}
>>> +
>>> +static int __init
>>> +scmi_dt_read_hyp_channel_addr(struct dt_device_node *scmi_node, u64 *addr,
>>> + u64 *size)
>>> +{
>>> + struct dt_device_node *shmem_node;
>>> + const __be32 *prop;
>>> +
>>> + prop = dt_get_property(scmi_node, "shmem", NULL);
>>> + if ( !prop )
>>> + return -EINVAL;
>>> +
>>> + shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
>>> + if ( IS_ERR_OR_NULL(shmem_node) )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: Device tree error, can't parse reserved memory %ld\n",
>>> + PTR_ERR(shmem_node));
>>> + return PTR_ERR(shmem_node);
>>> + }
>>> +
>>> + return dt_device_get_address(shmem_node, 0, addr, size);
>>> +}
>>> +
>>> +/*
>>> + * Handle Dom0 SCMI specific DT nodes
>>> + *
>>> + * Make a decision on copying SCMI specific nodes into Dom0 device tree.
>>> + * For SCMI multi-agent case:
>>> + * - shmem nodes will not be copied and generated instead if SCMI
>>> + * is enabled for Dom0
>>> + * - scmi node will be copied if SCMI is enabled for Dom0
>>> + */
>>> +static bool scmi_dt_handle_node(struct domain *d, struct dt_device_node *node)
>>> +{
>>> + static const struct dt_device_match skip_matches[] __initconst = {
>>> + DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
>>> + { /* sentinel */ },
>>> + };
>>> + static const struct dt_device_match scmi_matches[] __initconst = {
>>> + DT_MATCH_PATH("/firmware/scmi"),
>>> + { /* sentinel */ },
>>> + };
>>> +
>>> + if ( !scmi_data.initialized )
>>> + return false;
>>> +
>>> + /* always drop shmem */
>>> + if ( dt_match_node(skip_matches, node) )
>>> + {
>>> + dt_dprintk(" Skip scmi shmem\n");
>>> + return true;
>>> + }
>>> +
>>> + /* drop scmi if not enabled */
>>> + if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
>>> + {
>>> + dt_dprintk(" Skip scmi node\n");
>>> + return true;
>>> + }
>>> +
>>> + return false;
>>> +}
>>> +
>>> +/*
>>> + * Finalize Dom0 SCMI specific DT nodes
>>> + *
>>> + * if SCMI is enabled for Dom0:
>>> + * - generate shmem node
>>> + * - map SCMI shmem MMIO into Dom0
>>> + */
>>> +static int scmi_dt_finalize(struct domain *d, void *fdt)
>>> +{
>>> + __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
>>> + struct scmi_channel *channel;
>>> + int nodeoffset;
>>> + __be32 *cells;
>>> + __be32 val;
>>> + char buf[64];
>>> + int res, rc;
>>> +
>>> + if ( !sci_domain_is_enabled(d) )
>>> + return 0;
>>> +
>>> + channel = d->arch.sci_data;
>>> +
>>> + /*
>>> + * Replace "arm,smc-id" with proper value assigned for Dom0 SCMI channel
>>> + */
>>> + nodeoffset = fdt_node_offset_by_compatible(fdt, -1, "arm,scmi-smc");
>>> + if ( nodeoffset < 0 )
>>> + return -ENODEV;
>>> +
>>> + cells = (__be32 *)&val;
>>> + dt_set_cell(&cells, 1, channel->func_id);
>>> + res = fdt_setprop_inplace(fdt, nodeoffset, "arm,smc-id", &val, sizeof(val));
>>> + if ( res )
>>> + return -EINVAL;
>>> +
>> Are you sure it is worth to go through all this trouble to modify FDT in
>> place when we could simply generate the DT node from scratch like we do
>> for example for the GIC? This seems to be more error prone as well. Is
>> generating it from scratch is really difficult? If it is difficult then OK.
>>
>>
>>> + /*
>>> + * All SCMI shmem nodes should be removed from Dom0 DT at this point, so
>>> + * the shmem node for Dom0 need to be generated from SCMI channel assigned
>>> + * to Dom0.
>>> + * The original SCMI shmem node from platform DT is used by Xen SCMI driver
>>> + * itself as privileged channel (agent_id=0) to manage other SCMI
>>> + * agents (domains).
>>> + */
>>> + snprintf(buf, sizeof(buf), "scmi-shmem@%lx", channel->paddr);
>>> +
>>> + res = fdt_begin_node(fdt, buf);
>>> + if ( res )
>>> + return res;
>>> +
>>> + res = fdt_property_string(fdt, "compatible", "arm,scmi-shmem");
>>> + if ( res )
>>> + return res;
>>> +
>>> + cells = ®[0];
>>> +
>>> + dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>>> + channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
>>> +
>>> + res = fdt_property(fdt, "reg", reg, sizeof(reg));
>>> + if ( res )
>>> + return res;
>>> +
>>> + res = fdt_property_cell(fdt, "phandle", scmi_data.shmem_phandle);
>>> + if ( res )
>>> + return res;
>>> +
>>> + res = fdt_end_node(fdt);
>>> + if ( res )
>>> + return res;
>>> +
>>> + /*
>>> + * Map SCMI shmem into Dom0 here as shmem nodes are excluded from
>>> + * generic Dom0 DT processing
>>> + */
>>> + res = iomem_permit_access(d, paddr_to_pfn(channel->paddr),
>>> + paddr_to_pfn(channel->paddr +
>>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>>> + if ( res )
>>> + return res;
>>> +
>>> + res = map_regions_p2mt(d, gaddr_to_gfn(channel->paddr),
>>> + PFN_UP(SCMI_SHMEM_MAPPED_SIZE),
>>> + maddr_to_mfn(channel->paddr), p2m_mmio_direct_nc);
>>> + if ( res )
>>> + {
>>> + rc = iomem_deny_access(d, paddr_to_pfn(channel->paddr),
>>> + paddr_to_pfn(channel->paddr +
>>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>>> + if ( rc )
>>> + printk(XENLOG_ERR "scmi: Unable to deny iomem access , err = %d\n",
>>> + rc);
>>> + }
>>> +
>>> + return res;
>>> +}
>>> +
>>> +static int scmi_assign_device(uint32_t agent_id, uint32_t device_id,
>>> + uint32_t flags)
>>> +{
>>> + struct scmi_msg_base_set_device_permissions_a2p tx;
>>> + struct scmi_channel *channel;
>>> + scmi_msg_header_t hdr;
>>> + int ret;
>>> +
>>> + channel = get_channel_by_id(HYP_CHANNEL);
>>> + if ( !channel )
>>> + return -EINVAL;
>>> +
>>> + hdr.id = SCMI_BASE_SET_DEVICE_PERMISSIONS;
>>> + hdr.type = 0;
>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>> +
>>> + tx.agent_id = agent_id;
>>> + tx.device_id = device_id;
>>> + tx.flags = flags;
>>> +
>>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>>> + if ( ret == -EOPNOTSUPP )
>>> + return 0;
>> Is it actually OK to pretend that everything worked if the return is
>> -EOPNOTSUPP? I mean that in this case can we assume that the device is
>> actually assigned anyway? Wouldn't follow up SCMI operations on this
>> device fail?
>>
>>
>>> + return ret;
>>> +}
>>> +
>>> +static int scmi_dt_assign_device(struct domain *d,
>>> + struct dt_phandle_args *ac_spec)
>>> +{
>>> + struct scmi_channel *agent_channel;
>>> + uint32_t scmi_device_id = ac_spec->args[0];
>>> + int ret;
>>> +
>>> + if ( !d->arch.sci_data )
>>> + return 0;
>>> +
>>> + /* The access-controllers is specified for DT dev, but it's not a SCMI */
>>> + if ( ac_spec->np != scmi_data.dt_dev )
>>> + return 0;
>> I wonder if this should be an error
>>
>>
>>> + agent_channel = d->arch.sci_data;
>>> +
>>> + spin_lock(&agent_channel->lock);
>>> +
>>> + ret = scmi_assign_device(agent_channel->agent_id, scmi_device_id,
>>> + SCMI_BASE_DEVICE_ACCESS_ALLOW);
>>> + if ( ret )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: could not assign dev for %pd agent:%d dev_id:%u (%d)",
>>> + d, agent_channel->agent_id, scmi_device_id, ret);
>>> + }
>>> +
>>> + spin_unlock(&agent_channel->lock);
>>> + return ret;
>>> +}
>>> +
>>> +static __init int collect_agents(struct dt_device_node *scmi_node)
>>> +{
>>> + const struct dt_device_node *chosen_node;
>>> + const __be32 *prop;
>>> + uint32_t len, i;
>>> +
>>> + chosen_node = dt_find_node_by_path("/chosen");
>>> + if ( !chosen_node )
>>> + {
>>> + printk(XENLOG_ERR "scmi: chosen node not found\n");
>>> + return -ENOENT;
>>> + }
>>> +
>>> + prop = dt_get_property(chosen_node, SCMI_SECONDARY_AGENTS, &len);
>>> + if ( !prop )
>>> + {
>>> + printk(XENLOG_WARNING "scmi: No %s property found\n",
>>> + SCMI_SECONDARY_AGENTS);
>>> + return -ENODEV;
>>> + }
>>> +
>>> + if ( len % (3 * sizeof(uint32_t)) )
>>> + {
>>> + printk(XENLOG_ERR "scmi: Invalid length of %s property: %d\n",
>>> + SCMI_SECONDARY_AGENTS, len);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + for ( i = 0; i < len / (3 * sizeof(uint32_t)); i++ )
>>> + {
>>> + uint32_t agent_id = be32_to_cpu(*prop++);
>>> + uint32_t smc_id = be32_to_cpu(*prop++);
>>> + uint32_t shmem_phandle = be32_to_cpu(*prop++);
>>> + struct dt_device_node *node = dt_find_node_by_phandle(shmem_phandle);
>>> + u64 addr, size;
>>> + int ret;
>>> +
>>> + if ( !node )
>>> + {
>>> + printk(XENLOG_ERR "scmi: Could not find shmem node for agent %d\n",
>>> + agent_id);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + ret = dt_device_get_address(node, 0, &addr, &size);
>>> + if ( ret )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: Could not read shmem address for agent %d: %d",
>>> + agent_id, ret);
>>> + return ret;
>>> + }
>>> +
>>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>>> + {
>>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + ret = PTR_RET(smc_create_channel(agent_id, smc_id, addr));
>>> + if ( ret )
>>> + {
>>> + printk(XENLOG_ERR "scmi: Could not create channel for agent %d: %d",
>>> + agent_id, ret);
>>> + return ret;
>>> + }
>>> +
>>> + printk(XENLOG_DEBUG "scmi: Agent %d SMC %X addr %lx\n", agent_id,
>>> + smc_id, addr);
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int scmi_domain_init(struct domain *d,
>>> + struct xen_domctl_createdomain *config)
>>> +{
>>> + struct scmi_channel *channel;
>>> + int ret;
>>> +
>>> + if ( !scmi_data.initialized )
>>> + return 0;
>>> +
>>> + /*
>>> + * Special case for Dom0 - the SCMI support is enabled basing on
>>> + * "dom0_sci_agent_id" Xen command line parameter
>>> + */
>>> + if ( is_hardware_domain(d) )
>>> + {
>>> + if ( opt_dom0_scmi_agent_id != SCMI_AGENT_ID_INVALID )
>>> + {
>>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>> + config->arch.arm_sci_agent_id = opt_dom0_scmi_agent_id;
>>> + }
>>> + else
>>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>>> + }
>>> +
>>> + if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE )
>>> + return 0;
>>> +
>>> + channel = acquire_scmi_channel(d, config->arch.arm_sci_agent_id);
>>> + if ( IS_ERR(channel) )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: Failed to acquire SCMI channel for agent_id %u: %ld\n",
>>> + config->arch.arm_sci_agent_id, PTR_ERR(channel));
>>> + return PTR_ERR(channel);
>>> + }
>>> +
>>> + printk(XENLOG_INFO
>>> + "scmi: Acquire channel id = 0x%x, domain_id = %d paddr = 0x%lx\n",
>>> + channel->agent_id, channel->domain_id, channel->paddr);
>>> +
>>> + /*
>>> + * Dom0 (if present) needs to have an access to the guest memory range
>>> + * to satisfy iomem_access_permitted() check in XEN_DOMCTL_iomem_permission
>>> + * domctl.
>> Ideally this should not be needed but I understand we don't have an
>> easy solution, I think we can go ahead with this for now.
>>
>>
>>> + */
>>> + if ( hardware_domain && !is_hardware_domain(d) )
>>> + {
>>> + ret = iomem_permit_access(hardware_domain, paddr_to_pfn(channel->paddr),
>>> + paddr_to_pfn(channel->paddr + PAGE_SIZE - 1));
>>> + if ( ret )
>>> + goto error;
>>> + }
>>> +
>>> + d->arch.sci_data = channel;
>>> + d->arch.sci_enabled = true;
>>> +
>>> + return 0;
>>> +
>>> +error:
>>> + relinquish_scmi_channel(channel);
>>> + return ret;
>>> +}
>>> +
>>> +int scmi_domain_sanitise_config(struct xen_domctl_createdomain *config)
>>> +{
>>> + if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
>>> + config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA )
>>> + {
>>> + dprintk(XENLOG_INFO, "scmi: Unsupported ARM_SCI type\n");
>>> + return -EINVAL;
>>> + }
>>> + else if ( config->arch.arm_sci_type ==
>>> + XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA &&
>>> + config->arch.arm_sci_agent_id == 0 )
>>> + {
>>> + dprintk(XENLOG_INFO,
>>> + "scmi: A zero ARM SCMI agent_id is not supported\n");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int scmi_relinquish_resources(struct domain *d)
>>> +{
>>> + int ret;
>>> + struct scmi_channel *channel, *agent_channel;
>>> + scmi_msg_header_t hdr;
>>> + struct scmi_msg_base_reset_agent_cfg_a2p tx;
>>> +
>>> + if ( !d->arch.sci_data )
>>> + return 0;
>>> +
>>> + agent_channel = d->arch.sci_data;
>>> +
>>> + spin_lock(&agent_channel->lock);
>>> + tx.agent_id = agent_channel->agent_id;
>>> + spin_unlock(&agent_channel->lock);
>>> +
>>> + channel = get_channel_by_id(HYP_CHANNEL);
>>> + if ( !channel )
>>> + {
>>> + printk(XENLOG_ERR
>>> + "scmi: Unable to get Hypervisor scmi channel for domain %d\n",
>>> + d->domain_id);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + hdr.id = SCMI_BASE_RESET_AGENT_CONFIGURATION;
>>> + hdr.type = 0;
>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>> +
>>> + tx.flags = 0;
>>> +
>>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>>> + if ( ret == -EOPNOTSUPP )
>>> + return 0;
>>> +
>>> + return ret;
>>> +}
>>> +
>>> +static void scmi_domain_destroy(struct domain *d)
>>> +{
>>> + struct scmi_channel *channel;
>>> +
>>> + if ( !d->arch.sci_data )
>>> + return;
>>> +
>>> + channel = d->arch.sci_data;
>>> + spin_lock(&channel->lock);
>>> +
>>> + relinquish_scmi_channel(channel);
>>> + printk(XENLOG_DEBUG "scmi: Free domain %d\n", d->domain_id);
>>> +
>>> + d->arch.sci_data = NULL;
>>> + d->arch.sci_enabled = true;
>>> +
>>> + spin_unlock(&channel->lock);
>>> +}
>>> +
>>> +static bool scmi_handle_call(struct cpu_user_regs *regs)
>>> +{
>>> + uint32_t fid = (uint32_t)get_user_reg(regs, 0);
>>> + struct scmi_channel *agent_channel;
>>> + struct domain *d = current->domain;
>>> + struct arm_smccc_res resp;
>>> + bool res = false;
>>> +
>>> + if ( !sci_domain_is_enabled(d) )
>>> + return false;
>>> +
>>> + agent_channel = d->arch.sci_data;
>>> + spin_lock(&agent_channel->lock);
>>> +
>>> + if ( agent_channel->func_id != fid )
>>> + {
>>> + res = false;
>>> + goto unlock;
>>> + }
>>> +
>>> + arm_smccc_1_1_smc(fid,
>>> + get_user_reg(regs, 1),
>>> + get_user_reg(regs, 2),
>>> + get_user_reg(regs, 3),
>>> + get_user_reg(regs, 4),
>>> + get_user_reg(regs, 5),
>>> + get_user_reg(regs, 6),
>>> + get_user_reg(regs, 7),
>>> + &resp);
>>> +
>>> + set_user_reg(regs, 0, resp.a0);
>>> + set_user_reg(regs, 1, resp.a1);
>>> + set_user_reg(regs, 2, resp.a2);
>>> + set_user_reg(regs, 3, resp.a3);
>>> + res = true;
>>> +unlock:
>>> + spin_unlock(&agent_channel->lock);
>>> +
>>> + return res;
>>> +}
>>> +
>>> +static const struct sci_mediator_ops scmi_ops = {
>>> + .domain_init = scmi_domain_init,
>>> + .domain_destroy = scmi_domain_destroy,
>>> + .relinquish_resources = scmi_relinquish_resources,
>>> + .handle_call = scmi_handle_call,
>>> + .dom0_dt_handle_node = scmi_dt_handle_node,
>>> + .dom0_dt_finalize = scmi_dt_finalize,
>>> + .domain_sanitise_config = scmi_domain_sanitise_config,
>>> + .assign_dt_device = scmi_dt_assign_device,
>>> +};
>>> +
>>> +static int __init scmi_check_smccc_ver(void)
>>> +{
>>> + if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
>>> + {
>>> + printk(XENLOG_WARNING
>>> + "scmi: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
>>> + return -ENOSYS;
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static __init int scmi_probe(struct dt_device_node *scmi_node, const void *data)
>>> +{
>>> + u64 addr, size;
>>> + int ret, i;
>>> + struct scmi_channel *channel, *agent_channel;
>>> + int n_agents;
>>> + scmi_msg_header_t hdr;
>>> + struct scmi_msg_base_attributes_p2a rx;
>>> +
>>> + ASSERT(scmi_node != NULL);
>>> +
>>> + INIT_LIST_HEAD(&scmi_data.channel_list);
>>> + spin_lock_init(&scmi_data.channel_list_lock);
>>> +
>>> + if ( !acpi_disabled )
>>> + {
>>> + printk(XENLOG_WARNING "scmi: is not supported when using ACPI\n");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + ret = scmi_check_smccc_ver();
>>> + if ( ret )
>>> + return ret;
>>> +
>>> + if ( !dt_property_read_u32(scmi_node, "arm,smc-id", &scmi_data.func_id) )
>>> + {
>>> + printk(XENLOG_ERR "scmi: unable to read smc-id from DT\n");
>>> + return -ENOENT;
>>> + }
>>> +
>>> + /* save shmem phandle and re-use it fro Dom0 DT shmem node */
>>> + if ( !dt_property_read_u32(scmi_node, "shmem", &scmi_data.shmem_phandle) )
>>> + {
>>> + printk(XENLOG_ERR "scmi: unable to read shmem phandle from DT\n");
>>> + return -ENOENT;
>>> + }
>>> +
>>> + ret = scmi_dt_read_hyp_channel_addr(scmi_node, &addr, &size);
>>> + if ( IS_ERR_VALUE(ret) )
>>> + return -ENOENT;
>>> +
>>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>>> + {
>>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + scmi_data.dt_dev = scmi_node;
>>> +
>>> + channel = smc_create_channel(HYP_CHANNEL, scmi_data.func_id, addr);
>>> + if ( IS_ERR(channel) )
>>> + goto out;
>>> +
>>> + ret = map_channel_memory(channel);
>>> + if ( ret )
>>> + goto out;
>>> +
>>> + channel->domain_id = DOMID_XEN;
>>> +
>>> + hdr.id = SCMI_BASE_PROTOCOL_ATTIBUTES;
>>> + hdr.type = 0;
>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>> +
>>> + ret = do_smc_xfer(channel, &hdr, NULL, 0, &rx, sizeof(rx));
>>> + if ( ret )
>>> + goto error;
>>> +
>>> + n_agents = SCMI_FIELD_GET(SCMI_BASE_ATTR_NUM_AGENT, rx.attributes);
>>> + printk(XENLOG_DEBUG "scmi: Got agent count %d\n", n_agents);
>>> +
>>> + ret = collect_agents(scmi_node);
>>> + if ( ret )
>>> + goto error;
>>> +
>>> + i = 1;
>>> +
>>> + list_for_each_entry(agent_channel, &scmi_data.channel_list, list)
>>> + {
>>> + struct scmi_msg_base_discover_agent_p2a da_rx;
>>> + struct scmi_msg_base_discover_agent_a2p da_tx;
>>> +
>>> + ret = map_channel_memory(agent_channel);
>>> + if ( ret )
>>> + goto error;
>>> +
>>> + hdr.id = SCMI_BASE_DISCOVER_AGENT;
>>> + hdr.type = 0;
>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>> +
>>> + da_tx.agent_id = agent_channel->agent_id;
>>> +
>>> + ret = do_smc_xfer(agent_channel, &hdr, &da_tx, sizeof(da_tx), &da_rx,
>>> + sizeof(da_rx));
>>> + if ( agent_channel->domain_id != DOMID_XEN )
>>> + unmap_channel_memory(agent_channel);
>>> + if ( ret )
>>> + goto error;
>>> +
>>> + printk(XENLOG_DEBUG "id=0x%x name=%s\n", da_rx.agent_id, da_rx.name);
>>> +
>>> + agent_channel->agent_id = da_rx.agent_id;
>> It is OK to set agent_channel->agent_id to the value provided by the
>> SCMI server, but if we are also taking the agent_channel->agent_id value
>> from the user via device tree, shouldn't we throw an error if there is a
>> mismatch?
>>
>> Or even better: can we avoid taking the value via device tree to make it
>> easier to configure?
>>
>>
>>> + if ( i > n_agents )
>>> + break;
>>> +
>>> + i++;
>>> + }
>>> +
>>> + ret = sci_register(&scmi_ops);
>>> + if ( ret )
>>> + {
>>> + printk(XENLOG_ERR "SCMI: mediator already registered (ret = %d)\n",
>>> + ret);
>>> + return ret;
>>> + }
>>> +
>>> + scmi_data.initialized = true;
>>> + goto out;
>>> +
>>> +error:
>>> + unmap_channel_memory(channel);
>>> + free_channel_list();
>>> +out:
>>> + return ret;
>>> +}
>>> +
>>> +static const struct dt_device_match scmi_smc_match[] __initconst = {
>>> + DT_MATCH_COMPATIBLE("arm,scmi-smc"),
>>> + { /* sentinel */ },
>>> +};
>>> +
>>> +DT_DEVICE_START(scmi_smc_ma, "SCMI SMC MEDIATOR", DEVICE_FIRMWARE)
>>> + .dt_match = scmi_smc_match,
>>> + .init = scmi_probe,
>>> +DT_DEVICE_END
>>> +
>>> +/*
>>> + * Local variables:
>>> + * mode: C
>>> + * c-file-style: "BSD"
>>> + * c-basic-offset: 4
>>> + * tab-width: 4
>>> + * indent-tabs-mode: nil
>>> + * End:
>>> + */
>>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
>>> index 095b1a23e3..30e46de6d7 100644
>>> --- a/xen/include/public/arch-arm.h
>>> +++ b/xen/include/public/arch-arm.h
>>> @@ -329,6 +329,7 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>>>
>>> #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE 0
>>> #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC 1
>>> +#define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA 2
>>>
>>> struct xen_arch_domainconfig {
>>> /* IN/OUT */
>>> @@ -355,6 +356,8 @@ struct xen_arch_domainconfig {
>>> uint32_t clock_frequency;
>>> /* IN */
>>> uint8_t arm_sci_type;
>>> + /* IN */
>>> + uint8_t arm_sci_agent_id;
>>> };
>>> #endif /* __XEN__ || __XEN_TOOLS__ */
>>>
>>> --
>>> 2.34.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-02 7:17 ` Bertrand Marquis
2025-06-12 12:03 ` Oleksii Moisieiev
@ 2025-06-12 12:10 ` Grygorii Strashko
2025-06-17 23:38 ` Stefano Stabellini
1 sibling, 1 reply; 67+ messages in thread
From: Grygorii Strashko @ 2025-06-12 12:10 UTC (permalink / raw)
To: Bertrand Marquis, Stefano Stabellini
Cc: Oleksii Moisieiev, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk
On 02.06.25 10:17, Bertrand Marquis wrote:
> Hi Stefano and Oleksii,
>
>> On 23 May 2025, at 22:06, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>
>> One question for Bertrand below
>>
>>
>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
>>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
>>> below.
>>>
>>> +-----------------------------------------+
>>> | |
>>> | EL3 TF-A SCMI |
>>> +-------+--+-------+--+-------+--+-------++
>>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
>>> +-----+-+ +---+---+ +--+----+ +---+---+
>>> smc-id0 | | | |
>>> agent0 | | | |
>>> +-----v--------+---------+-----------+----+
>>> | | | | |
>>> | | | | |
>>> +--------------+---------+-----------+----+
>>> smc-id1 | smc-id2| smc-idX|
>>> agent1 | agent2 | agentX |
>>> | | |
>>> +----v---+ +--v-----+ +--v-----+
>>> | | | | | |
>>> | Dom0 | | Dom1 | | DomX |
>>> | | | | | |
>>> | | | | | |
>>> +--------+ +--------+ +--------+
>>>
>>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
>>> memory transport for every Agent in the system.
>>>
>>> The SCMI Agent transport channel defined by pair:
>>> - smc-id: SMC/HVC id used for Doorbell
>>> - shmem: shared memory for messages transfer, Xen page aligned,
>>> p2m_mmio_direct_nc.
>>>
>>> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
>>> multi-agent functionality under Xen:
>>> - Xen manegement agent: trusted agents that accesses to the Base Protocol
>>> commands to configure agent specific permissions
>>> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
>>> allowed direct HW access. At least one OSPM VM agent has to be provided
>>> by FW if HW is handled only by Dom0 or Driver Domain.
>>>
>>> The EL3 SCMI FW expected to implement following Base protocol messages:
>>> - BASE_DISCOVER_AGENT
>>> - BASE_RESET_AGENT_CONFIGURATION (optional)
>>> - BASE_SET_DEVICE_PERMISSIONS (optional)
>>>
>>> The SCI SCMI SMC multi-agent driver implements following functionality:
>>> - It's initialized based on the Host DT SCMI node (only one SCMI interface
>>> is supported) which describes Xen management agent SCMI interface.
>>>
>>> scmi_shm_0 : sram@47ff0000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff0000 0x0 0x1000>;
>>> };
>>> firmware {
>>> scmi: scmi {
>>> compatible = "arm,scmi-smc";
>>> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
>>
>> some extra spaces, it might be a copy/paste error
>>
>>
>>> \#address-cells = < 1>;
>>> \#size-cells = < 0>;
>>> \#access-controller - cells = < 1>;
>>> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
>>>
>>> protocol@X{
>>> };
>>> };
>>> };
>>>
>>> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
>>> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
>>>
>>> chosen {
>>> xen,scmi-secondary-agents = <
>>> 1 0x82000003 &scmi_shm_1
>>> 2 0x82000004 &scmi_shm_2
>>> 3 0x82000005 &scmi_shm_3
>>> 4 0x82000006 &scmi_shm_4>;
>>> }
>>>
>>> /{
>>> scmi_shm_1: sram@47ff1000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff1000 0x0 0x1000>;
>>> };
>>> scmi_shm_2: sram@47ff2000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff2000 0x0 0x1000>;
>>> };
>>> scmi_shm_3: sram@47ff3000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x47ff3000 0x0 0x1000>;
>>> };
>>> }
>>> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
>>> this agent_id.
>>>
>>> Note that Xen is the only one entry in the system which need to know
>>> about SCMI multi-agent support.
>>>
>>> - It implements the SCI subsystem interface required for configuring and
>>> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
>>> SCMI functionality for domain it has to be configured with unique supported
>>> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
>>> [smc-id, shmem] defined for this SCMI Agent_id.
>>> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
>>> -- zero-copy, the guest domain puts SCMI message in shmem;
>>> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
>>> -- the Xen driver catches exception, do checks and synchronously forwards
>>> it to EL3 FW.
>>> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
>>> management agent channel on domain destroy event. This allows to reset
>>> resources used by domain and so implement use-case like domain reboot.
>>>
>>> Dom0 Enable SCMI SMC:
>>> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
>>> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
>>> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
>>> node according to assigned agent_id.
>>>
>>> Guest domains enable SCMI SMC:
>>> - xl.cfg: add configuration option as below
>>>
>>> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
>>>
>>> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
>>> the domain, for example:
>>>
>>> iomem = [
>>> "47ff2,1@22001",
>>> ]
>>
>> Looking at the code and the configuration options, it looks like it is
>> possible to map a scmi-shmem channel at a different address for the
>> guest. It seems like it would work. Is that correct?
>>
>>
>>> - DT: add SCMI nodes to the Driver domain partial device tree as in the
>>> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
>>>
>>> passthrough {
>>> scmi_shm_0: sram@22001000 {
>>> compatible = "arm,scmi-shmem";
>>> reg = <0x0 0x22001000 0x0 0x1000>;
>>> };
>>>
>>> firmware {
>>> compatible = "simple-bus";
>>> scmi: scmi {
>>> compatible = "arm,scmi-smc";
>>> arm,smc-id = <0x82000004>;
>>> shmem = <&scmi_shm_0>;
>>> ...
>>> }
>>> }
>>> }
>>>
>>> SCMI "4.2.1.1 Device specific access control"
>>>
>>> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
>>> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
>>> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
>>> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
>>> to the Xen SCMI.
>>>
>>> &i2c1 {
>>> access-controllers = <&scmi 0>;
>>> };
>>>
>>> The Dom0 and dom0less domains DT devices will be processed automatically through
>>> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
>>> shell be used:
>>>
>>> dtdev = [
>>> "/soc/i2c@e6508000",
>>> ]
>>>
>>> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
>>>
>>> [1] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
>>> [2] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
>>> Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
>>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>>
>> Thanks for the long explanation, great work! I am really looking forward
>> to have this feature in the tree soon.
>>
>>
>>> ---
>>>
>>> Changes in v4:
>>> - toolstack comments from Anthony PERARD
>>> - added dom0less support
>>> - added doc for "xen,scmi-secondary-agents"
>>>
>>> docs/man/xl.cfg.5.pod.in | 13 +
>>> docs/misc/arm/device-tree/booting.txt | 60 ++
>>> docs/misc/xen-command-line.pandoc | 9 +
>>> tools/libs/light/libxl_arm.c | 4 +
>>> tools/libs/light/libxl_types.idl | 4 +-
>>> tools/xl/xl_parse.c | 12 +
>>> xen/arch/arm/dom0less-build.c | 11 +
>>> xen/arch/arm/domain_build.c | 3 +-
>>> xen/arch/arm/firmware/Kconfig | 11 +
>>> xen/arch/arm/firmware/Makefile | 1 +
>>> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
>>> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
>>> xen/arch/arm/firmware/scmi-shmem.h | 45 +
>>> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
>>> xen/include/public/arch-arm.h | 3 +
>>> 15 files changed, 1371 insertions(+), 2 deletions(-)
>>> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
>>> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
>>>
>>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>>> index 1ccf50b8ea..302c46d8bc 100644
>>> --- a/docs/man/xl.cfg.5.pod.in
>>> +++ b/docs/man/xl.cfg.5.pod.in
>>> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
>>> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
>>> option.
>>>
>>> +=item B<scmi_smc_multiagent>
>>> +
>>> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>>> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
>>> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
>>> +specified for the guest.
>>> +
>>> =back
>>>
>>> +=item B<agent_id=NUMBER>
>>> +
>>> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
>>> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
>>> +existing on a single host must be unique and in the range [1..255].
>>> +
>>> =back
>>>
>>> =back
>>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
>>> index 8943c04173..c8923ab8b2 100644
>>> --- a/docs/misc/arm/device-tree/booting.txt
>>> +++ b/docs/misc/arm/device-tree/booting.txt
>>> @@ -296,6 +296,20 @@ with the following properties:
>>> Should be used together with dom0_scmi_smc_passthrough Xen command line
>>> option.
>>>
>>> + - "scmi_smc_multiagent"
>>> +
>>> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>>> + SMC calls forwarding from domain to the EL3 firmware (like ARM
>>> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
>>> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
>>> + property.
>>> +
>>> +- "xen,sci_agent_id"
>>> +
>>> + Specifies a non-zero ARM SCI agent id for the guest. This option is
>>> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
>>> + the guest. The agent ids of guest must be unique and in the range [1..255].
>>> +
>>> Under the "xen,domain" compatible node, one or more sub-nodes are present
>>> for the DomU kernel and ramdisk.
>>>
>>> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
>>> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
>>> guest physical address space. DomU1 is explicitly defined as the owner domain,
>>> and DomU2 is the borrower domain.
>>> +
>>> +SCMI SMC multi-agent support
>>> +============================
>>> +
>>> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
>>> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
>>> +according to the SCMI compliant EL3 Firmware specification with
>>> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
>>> +the top-level "chosen" node:
>>> +
>>> +- xen,scmi-secondary-agents
>>> +
>>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
>>> + available for Xen. Each Agent defined as triple consisting of:
>>> + SCMI agent_id,
>>> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
>>> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
>>> +
>>> +As an example:
>>> +
>>> +chosen {
>>> + xen,scmi-secondary-agents = <
>>> + 1 0x82000003 &scmi_shm_1
>>> + 2 0x82000004 &scmi_shm_2
>>> + 3 0x82000005 &scmi_shm_3
>>> + 4 0x82000006 &scmi_shm_4>;
>>> +}
>>
>> NIT: it should be };
>>
>> Looking at scmi_probe, collect_agents, and the following SCMI
>> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
>> information?
>>
>> It looks like we can discover the agend_ids for every channel, I guess
>> what we need to know is the shmem location for every channel? But the
>> full list of shmem channel is available below thanks to the scmi-shmem
>> nodes.
>>
>> So, we have the list of scmi-shmem anyway, and we can probe the
>> agent_id. The only parameter left is the smc_id/func_id.
>>
>> Or maybe smc_id/func_id can be calculated from agent_id?
>>
>> I am asking mostly because if a user is supposed to add this
>> xen,scmi-secondary-agents property, where are they supposed to find the
>> smc_id/func_id information?
>>
>> It is important that we write down in this document how the user is
>> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
>>
>>
>>> +/{
>>> + scmi_shm_1: sram@47ff1000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>>> + };
>>> + scmi_shm_2: sram@47ff2000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>>> + };
>>> + scmi_shm_3: sram@47ff3000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>>> + };
>>> + scmi_shm_3: sram@47ff4000 {
>>> + compatible = "arm,scmi-shmem";
>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>>> + };
>>
>> Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
>> Or are under /firmware? Or are they under /chosen?
>>
>> I take they are under the top level node together with scmi_shm_0?
>>
>> Can you please also clarify in the document as well?
>>
>>
>>> +}
>>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
>>> index 8e50f6b7c7..bc3c64d6ec 100644
>>> --- a/docs/misc/xen-command-line.pandoc
>>> +++ b/docs/misc/xen-command-line.pandoc
>>> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
>>> SCMI nodes removed from Dom0/hwdom device tree.
>>> (for example, thin Dom0 with Driver domain use-case).
>>>
>>> +### dom0_scmi_agent_id (ARM)
>>> +> `= <integer>`
>>> +
>>> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
>>> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
>>> +The SCMI will be disabled for Dom0 if this option is not specified
>>> +(for example, thin Dom0 or dom0less use-cases).
>>> +The agent ids of domains existing on a single host must be unique.
>>> +
>>> ### dtuart (ARM)
>>>> `= path [:options]`
>>>
>>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>>> index 28ba9eb787..7712f53cd4 100644
>>> --- a/tools/libs/light/libxl_arm.c
>>> +++ b/tools/libs/light/libxl_arm.c
>>> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
>>> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>> break;
>>> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
>>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
>>> + break;
>>> default:
>>> LOG(ERROR, "Unknown ARM_SCI type %d",
>>> d_config->b_info.arch_arm.arm_sci.type);
>>> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
>>> index aa2190ab5b..11e31ce786 100644
>>> --- a/tools/libs/light/libxl_types.idl
>>> +++ b/tools/libs/light/libxl_types.idl
>>> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
>>>
>>> libxl_arm_sci_type = Enumeration("arm_sci_type", [
>>> (0, "none"),
>>> - (1, "scmi_smc")
>>> + (1, "scmi_smc"),
>>> + (2, "scmi_smc_multiagent")
>>> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
>>>
>>> libxl_arm_sci = Struct("arm_sci", [
>>> ("type", libxl_arm_sci_type),
>>> + ("agent_id", uint8)
>>> ])
>>>
>>> libxl_rdm_reserve = Struct("rdm_reserve", [
>>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>>> index bd22be9d33..81aa3797e3 100644
>>> --- a/tools/xl/xl_parse.c
>>> +++ b/tools/xl/xl_parse.c
>>> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
>>> }
>>> }
>>>
>>> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
>>> + unsigned long val = parse_ulong(oparg);
>>> +
>>> + if (!val || val > 255) {
>>> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
>>> + val);
>>> + ret = ERROR_INVAL;
>>> + goto parse_error;
>>> + }
>>> + arm_sci->agent_id = val;
>>> + }
>>> +
>>> ptr = strtok(NULL, ",");
>>> }
>>>
>>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>>> index 0a00f03a25..43d21eb889 100644
>>> --- a/xen/arch/arm/dom0less-build.c
>>> +++ b/xen/arch/arm/dom0less-build.c
>>> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>>> else if ( !strcmp(sci_type, "scmi_smc") )
>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
>>> + {
>>> + uint32_t agent_id = 0;
>>> +
>>> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
>>> + !agent_id )
>>
>> shouldn't we check that agent_id <= 255 ?
>>
>>
>>> + return -EINVAL;
>>> +
>>> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>> + d_cfg->arch.arm_sci_agent_id = agent_id;
>>> + }
>>> else
>>> {
>>> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index 36d28b52a4..0c9274a2b3 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
>>> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
>>> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
>>> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
>>> continue;
>>>
>>> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") )
>>> diff --git a/xen/arch/arm/firmware/Kconfig b/xen/arch/arm/firmware/Kconfig
>>> index 5c5f0880c4..6b051c8ada 100644
>>> --- a/xen/arch/arm/firmware/Kconfig
>>> +++ b/xen/arch/arm/firmware/Kconfig
>>> @@ -29,6 +29,17 @@ config SCMI_SMC
>>> driver domain.
>>> Use with EL3 firmware which supports only single SCMI OSPM agent.
>>>
>>> +config SCMI_SMC_MA
>>> + bool "Enable ARM SCMI SMC multi-agent driver"
>>> + select ARM_SCI
>>> + help
>>> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
>>> + to EL3 firmware (TF-A) which supports multi-agent feature.
>>> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
>>> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
>>> + allowed platform resources through dedicated SMC/HVC Shared memory based
>>> + transport.
>>> +
>>> endchoice
>>>
>>> endmenu
>>> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
>>> index 71bdefc24a..37927e690e 100644
>>> --- a/xen/arch/arm/firmware/Makefile
>>> +++ b/xen/arch/arm/firmware/Makefile
>>> @@ -1,2 +1,3 @@
>>> obj-$(CONFIG_ARM_SCI) += sci.o
>>> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
>>> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
>>> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
>>> new file mode 100644
>>> index 0000000000..3f4b9c5d6b
>>> --- /dev/null
>>> +++ b/xen/arch/arm/firmware/scmi-proto.h
>>> @@ -0,0 +1,164 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * Arm System Control and Management Interface definitions
>>> + * Version 3.0 (DEN0056C)
>>> + *
>>> + * Copyright (c) 2024 EPAM Systems
>>> + */
>>> +
>>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>>> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>>
>> NIT: ARM_FIRMWARE_SCMI_PROTO_H
>>
>>
>>> +#include <xen/stdint.h>
>>> +
>>> +#define SCMI_SHORT_NAME_MAX_SIZE 16
>>> +
>>> +/* SCMI status codes. See section 4.1.4 */
>>> +#define SCMI_SUCCESS 0
>>> +#define SCMI_NOT_SUPPORTED (-1)
>>> +#define SCMI_INVALID_PARAMETERS (-2)
>>> +#define SCMI_DENIED (-3)
>>> +#define SCMI_NOT_FOUND (-4)
>>> +#define SCMI_OUT_OF_RANGE (-5)
>>> +#define SCMI_BUSY (-6)
>>> +#define SCMI_COMMS_ERROR (-7)
>>> +#define SCMI_GENERIC_ERROR (-8)
>>> +#define SCMI_HARDWARE_ERROR (-9)
>>> +#define SCMI_PROTOCOL_ERROR (-10)
>>> +
>>> +/* Protocol IDs */
>>> +#define SCMI_BASE_PROTOCOL 0x10
>>> +
>>> +/* Base protocol message IDs */
>>> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
>>> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
>>> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
>>> +#define SCMI_BASE_DISCOVER_AGENT 0x7
>>> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
>>> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
>>> +
>>> +typedef struct scmi_msg_header {
>>> + uint8_t id;
>>> + uint8_t type;
>>> + uint8_t protocol;
>>> + uint32_t status;
>>> +} scmi_msg_header_t;
>>> +
>>> +/* Table 2 Message header format */
>>> +#define SCMI_HDR_ID GENMASK(7, 0)
>>> +#define SCMI_HDR_TYPE GENMASK(9, 8)
>>> +#define SCMI_HDR_PROTO GENMASK(17, 10)
>>> +
>>> +#define SCMI_FIELD_GET(_mask, _reg) \
>>> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
>>> +#define SCMI_FIELD_PREP(_mask, _val) \
>>> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
>>> +
>>> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
>>> +{
>>> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
>>> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
>>> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
>>> +}
>>> +
>>> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
>>> +{
>>> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
>>> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
>>> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
>>> +}
>>> +
>>> +static inline int scmi_to_xen_errno(int scmi_status)
>>> +{
>>> + if ( scmi_status == SCMI_SUCCESS )
>>> + return 0;
>>> +
>>> + switch ( scmi_status )
>>> + {
>>> + case SCMI_NOT_SUPPORTED:
>>> + return -EOPNOTSUPP;
>>> + case SCMI_INVALID_PARAMETERS:
>>> + return -EINVAL;
>>> + case SCMI_DENIED:
>>> + return -EACCES;
>>> + case SCMI_NOT_FOUND:
>>> + return -ENOENT;
>>> + case SCMI_OUT_OF_RANGE:
>>> + return -ERANGE;
>>> + case SCMI_BUSY:
>>> + return -EBUSY;
>>> + case SCMI_COMMS_ERROR:
>>> + return -ENOTCONN;
>>> + case SCMI_GENERIC_ERROR:
>>> + return -EIO;
>>> + case SCMI_HARDWARE_ERROR:
>>> + return -ENXIO;
>>> + case SCMI_PROTOCOL_ERROR:
>>> + return -EBADMSG;
>>> + default:
>>> + return -EINVAL;
>>> + }
>>> +}
>>> +
>>> +/* PROTOCOL_VERSION */
>>> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
>>> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
>>> +
>>> +struct scmi_msg_prot_version_p2a {
>>> + uint32_t version;
>>> +} __packed;
>>> +
>>> +/* BASE PROTOCOL_ATTRIBUTES */
>>> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
>>> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
>>> +
>>> +struct scmi_msg_base_attributes_p2a {
>>> + uint32_t attributes;
>>> +} __packed;
>>> +
>>> +/*
>>> + * BASE_DISCOVER_AGENT
>>> + */
>>> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
>>> +
>>> +struct scmi_msg_base_discover_agent_a2p {
>>> + uint32_t agent_id;
>>> +} __packed;
>>> +
>>> +struct scmi_msg_base_discover_agent_p2a {
>>> + uint32_t agent_id;
>>> + char name[SCMI_SHORT_NAME_MAX_SIZE];
>>> +} __packed;
>>> +
>>> +/*
>>> + * BASE_SET_DEVICE_PERMISSIONS
>>> + */
>>> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
>>> +
>>> +struct scmi_msg_base_set_device_permissions_a2p {
>>> + uint32_t agent_id;
>>> + uint32_t device_id;
>>> + uint32_t flags;
>>> +} __packed;
>>> +
>>> +/*
>>> + * BASE_RESET_AGENT_CONFIGURATION
>>> + */
>>> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
>>> +
>>> +struct scmi_msg_base_reset_agent_cfg_a2p {
>>> + uint32_t agent_id;
>>> + uint32_t flags;
>>> +} __packed;
>>> +
>>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
>>> +
>>> +/*
>>> + * Local variables:
>>> + * mode: C
>>> + * c-file-style: "BSD"
>>> + * c-basic-offset: 4
>>> + * tab-width: 4
>>> + * indent-tabs-mode: nil
>>> + * End:
>>> + */
>>> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
>>> new file mode 100644
>>> index 0000000000..dd613ee0b5
>>> --- /dev/null
>>> +++ b/xen/arch/arm/firmware/scmi-shmem.c
>>> @@ -0,0 +1,173 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>>> + *
>>> + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
>>> + * Copyright (c) 2025 EPAM Systems
>>> + */
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +
>>> +#include <asm/io.h>
>>> +#include <xen/err.h>
>>> +
>>> +#include "scmi-proto.h"
>>> +#include "scmi-shmem.h"
>>
>> This code is written more generically than the description implies. If
>> we only want to make SMC calls to TF-A on EL3 and exchange data with it
>> over shared memory, then I think:
>> - we don't need the __iomem tag, as there is no MMIO
>
> agree
>
>> - we only need a DMB, not a DSB (readl and writel imply DSB, use only
>> readl_relaxed and writel_relaxed)
>
> agree
>
>>
>> On the other hand, if we also want to handle the case where the SCMI
>> server could be on a separate co-processor, then what this code is doing
>> is not sufficient because we also need a dcache flush, in addition to
>> the DSB.
>>
>> Bertrand, can you double-check?
>
> If we want to handle a case where the memory is accessible to a coprocessor
> but there is no cache coherency, we need to flush the dcache definitely.
>
> Seeing the amount of data here, I do agree with Stefano that it would be a good
> idea to make the provision to flush the data cache in all cases. Even if the data
> is accessed by a secure partition or the firmware coherently, flushing in all cases
> would have very limited performance impact here.
>
> There is the other solution to have some kind of parameter to say if the accessor
> has coherent cache access but I do not think the performance impact here would
> justify such a complexity.
>
The SCMI shmem expected to be mapped as MT_NON_CACHEABLE in all cases.
The Linux does devm_ioremap() -> ioremap() ->
(ARM64) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
There is also note in docs:
"+- shmem: shared memory for messages transfer, **Xen page aligned** with mapping``p2m_mmio_direct_nc``."
In the case of SCP - the SCMI shmem can be actually be in SRAM.
So, are you sure cache manipulations are required here?
--
Best regards,
-grygorii
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-12 11:42 ` Oleksii Moisieiev
@ 2025-06-12 13:10 ` Grygorii Strashko
2025-06-18 0:04 ` Stefano Stabellini
1 sibling, 0 replies; 67+ messages in thread
From: Grygorii Strashko @ 2025-06-12 13:10 UTC (permalink / raw)
To: Oleksii Moisieiev, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk
On 12.06.25 14:42, Oleksii Moisieiev wrote:
> Hi Stefano,
>
> I'm very sorry for a long silence. Please see my answers below:
>
> On 22/05/2025 03:25, Stefano Stabellini wrote:
>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>
>>> Add chained handling of assigned DT devices to support access-controller
>>> functionality through SCI framework, so DT device assign request can be
>>> passed to FW for processing and enabling VM access to requested device
>>> (for example, device power management through FW interface like SCMI).
>>>
>>> The SCI access-controller DT device processing is chained after IOMMU
>>> processing and expected to be executed for any DT device regardless of its
>>> protection by IOMMU (or if IOMMU is disabled).
>>>
>>> This allows to pass not only IOMMU protected DT device through
>>> xl.cfg:"dtdev" property for processing:
>>>
>>> dtdev = [
>>> "/soc/video@e6ef0000", <- IOMMU protected device
>>> "/soc/i2c@e6508000", <- not IOMMU protected device
>>> ]
>>>
>>> The change is done in two parts:
>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>>> not fail if DT device is not protected by IOMMU
>>> 2) add chained call to sci_do_domctl() in do_domctl()
>>>
>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>> ---
>>>
>>>
>>>
>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>>> xen/common/domctl.c | 19 +++++++++++++
>>> xen/drivers/passthrough/device_tree.c | 6 ++++
>>> 4 files changed, 76 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>> index e1522e10e2..8efd541c4f 100644
>>> --- a/xen/arch/arm/firmware/sci.c
>>> +++ b/xen/arch/arm/firmware/sci.c
>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>> return 0;
>>> }
>>>
>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>> +{
>>> + struct dt_device_node *dev;
>>> + int ret = 0;
>>> +
>>> + switch ( domctl->cmd )
>>> + {
>>> + case XEN_DOMCTL_assign_device:
>>> + ret = -EOPNOTSUPP;
>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>
> The -EOPNOTSUPP code is used because this is part of a chained call after
> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
> XEN_DOMCTL_assign_device
> call is expected to handle any DT device, regardless of whether the DT
> device is
> protected by an IOMMU or if the IOMMU is disabled.
> The following cases are considered:
>
> 1. IOMMU Protected Device (Success)
>
> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
> we continue
> processing the DT device by calling sci_do_domctl.
>
> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>
> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
> disabled,
> we still proceed to call sci_do_domctl.
>
> 3. Error from iommu_do_domctl (Fail State)
>
> If iommu_do_domctl returns any error, the system enters a fail state, and
> sci_do_domctl is not called.
>
> 4. -EOPNOTSUPP from sci_do_domctl
>
> If sci_do_domctl returns -EOPNOTSUPP, this indicates one of the following:
> - The provided device is not a DT device.
> - There is no cur_mediator available (indicating that the SCI subsystem
> is enabled
> in the configuration, but no mediator was provided).
> - The current mediator does not support assign_dt_device (this is
> expected to be changed;
> see below for details).
> In this case, -EOPNOTSUPP is returned but will be ignored, and the
> original return value from iommu_do_domctl will be used as the final result.
>
> 5. Return Code from sci_do_domctl
>
> If sci_do_domctl returns 0 (success) or an error code (failure),
> the return value from iommu_do_domctl is overridden, and the result from
> sci_do_domctl is returned.
> Note: -EOPNOTSUPP from iommu_do_domctl will also be overridden since
> step 2 was successfully completed (or failed).
>>> + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT )
>>> + break;
>> this one
>>
>>> + if ( !cur_mediator )
>>> + break;
>> this one
>>
>>> + if ( !cur_mediator->assign_dt_device )
>>> + break;
>> and also this one? It seems more like an -EINVAL as the caller used a
>> wrong parameter?
>
> I think you are right that this case should return -EINVAL because we
> should fail if mediator
>
> without implemented mandatory features was provided. Will be fixed.
>
>>> + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
>>> + domctl->u.assign_device.u.dt.size, &dev);
>>> + if ( ret )
>>> + return ret;
>>> +
>>> + ret = sci_assign_dt_device(d, dev);
>>> + if ( ret )
>>> + break;
>>> +
>>> + break;
>>> + default:
>>> + /* do not fail here as call is chained with iommu handling */
>> It looks like this should be an error
>>
>>
>>> + break;
>>> + }
>>> +
>>> + return ret;
>>> +}
>>> +
>>> static int __init sci_init(void)
>>> {
>>> struct dt_device_node *np;
>>> diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
>>> index 71fb54852e..b8d1bc8a62 100644
>>> --- a/xen/arch/arm/include/asm/firmware/sci.h
>>> +++ b/xen/arch/arm/include/asm/firmware/sci.h
>>> @@ -146,6 +146,14 @@ int sci_dt_finalize(struct domain *d, void *fdt);
>>> * control" functionality.
>>> */
>>> int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
>>> +
>>> +/*
>>> + * SCI domctl handler
>>> + *
>>> + * Only XEN_DOMCTL_assign_device is handled for now.
>>> + */
>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
>>> #else
>>>
>>> static inline bool sci_domain_is_enabled(struct domain *d)
>>> @@ -195,6 +203,12 @@ static inline int sci_assign_dt_device(struct domain *d,
>>> return 0;
>>> }
>>>
>>> +static inline int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>> +{
>>> + return 0;
>>> +}
>>> +
>>> #endif /* CONFIG_ARM_SCI */
>>>
>>> #endif /* __ASM_ARM_SCI_H */
>>> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
>>> index 05abb581a0..a74ee92067 100644
>>> --- a/xen/common/domctl.c
>>> +++ b/xen/common/domctl.c
>>> @@ -27,6 +27,7 @@
>>> #include <xen/vm_event.h>
>>> #include <xen/monitor.h>
>>> #include <asm/current.h>
>>> +#include <asm/firmware/sci.h>
>>> #include <asm/irq.h>
>>> #include <asm/page.h>
>>> #include <asm/p2m.h>
>>> @@ -851,6 +852,24 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>> case XEN_DOMCTL_deassign_device:
>>> case XEN_DOMCTL_get_device_group:
>>> ret = iommu_do_domctl(op, d, u_domctl);
>>> +
>>> + if ( !ret || ret == -EOPNOTSUPP )
>> It is better to invert the check:
>>
>> if ( ret < 0 && ret != -EOPNOTSUPP )
>> return ret;
> +
>>> + {
>>> + int ret1;
>>> + /*
>>> + * Add chained handling of assigned DT devices to support
>>> + * access-controller functionality through SCI framework, so
>>> + * DT device assign request can be passed to FW for processing and
>>> + * enabling VM access to requested device.
>>> + * The access-controller DT device processing is chained after IOMMU
>>> + * processing and expected to be executed for any DT device
>>> + * regardless if DT device is protected by IOMMU or not (or IOMMU
>>> + * is disabled).
>>> + */
>>> + ret1 = sci_do_domctl(op, d, u_domctl);
>>> + if ( ret1 != -EOPNOTSUPP )
>>> + ret = ret1;
>>> + }
>>> break;
>>>
>>> case XEN_DOMCTL_get_paging_mempool_size:
>>> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
>>> index 075fb25a37..2624767e51 100644
>>> --- a/xen/drivers/passthrough/device_tree.c
>>> +++ b/xen/drivers/passthrough/device_tree.c
>>> @@ -318,6 +318,12 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
>>> break;
>>> }
>>>
>>> + if ( !dt_device_is_protected(dev) )
>>> + {
>>> + ret = 0;
>>> + break;
>>> + }
>> I am concerned about this: previously we would call
>> iommu_assign_dt_device and the same check at the beginning of
>> iommu_assign_dt_device would return -EINVAL. Now it is a success.
>>
>> I am not sure this is appropriate. I wonder if instead we should:
>>
>> - remove this chunk from the patch
>> - change the return error for !dt_device_is_protected at the top of
>> iommu_assign_dt_device from -EINVAL to -EOPNOTSUPP
>> - this would fall into the same ret != -EOPNOTSUPP check after
>> iommu_do_domctl
We need to be careful here :(
The reason for this change, in it's current form is to make this part of
the code to work the same way as other parts [1] and [2] where IOMMU is configured (yep, and get what we need):
[1] xen/arch/arm/device.c handle_device()
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/device.c;h=5e1c1cc326ac04d30e7158905fead9b41c719fc0;hb=HEAD#l287
which is used to create hwdom(dom0) or apply overlays and called for every "own" node (!"xen,passthrough").
The "own" node may not be IOMMU protected.
[2] xen/common/device-tree/dom0less-build.c handle_passthrough_prop()
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/common/device-tree/dom0less-build.c;h=3d503c697337a4cc6df35403b9a2595b40089dae;hb=HEAD#l224
which is used for "dom0less" devices pass-through (arm/ppc) and called for every node with "xen,path" property,
but this path is intended to perform not only IOMMU, but also IRQs configuration.
The DT node, pointed by "xen,path", may not be IOMMU protected.
In both above cases call pattern is:
res = iommu_add_dt_device(node);
if ( res < 0 )
return res;
if ( !dt_device_is_protected(node) )
return 0; //no failure
res = iommu_assign_dt_device(domain, node);
if ( res )
return res;
And only when XEN_DOMCTL_assign_device (iommu_do_dt_domctl()) is handled the call pattern is
different and there is no check for dt_device_is_protected().
>
> That's a good point. I think we should do the same for
>
> > if ( !is_iommu_enabled(d) )
>
> > return -EINVAL;
>
> because in this case we should process sci as well. I will do the change
--
Best regards,
-grygorii
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-12 12:03 ` Oleksii Moisieiev
@ 2025-06-17 23:22 ` Stefano Stabellini
2025-06-17 23:56 ` Stefano Stabellini
2025-06-19 16:15 ` Oleksii Moisieiev
0 siblings, 2 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-17 23:22 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: Stefano Stabellini, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Julien Grall, Michal Orzel, Roger Pau Monné,
Volodymyr Babchuk, Grygorii Strashko
[-- Attachment #1: Type: text/plain, Size: 70007 bytes --]
On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> Hi Stefano,
>
> I'm very sorry for a long silence. Please see my answers below:
>
>
> On 23/05/2025 23:06, Stefano Stabellini wrote:
> > One question for Bertrand below
> >
> >
> > On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> >> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
> >> (TF-A) which provides SCMI interface with multi-agnet support, as shown
> >> below.
> >>
> >> +-----------------------------------------+
> >> | |
> >> | EL3 TF-A SCMI |
> >> +-------+--+-------+--+-------+--+-------++
> >> |shmem0 | |shmem1 | |shmem2 | |shmemX |
> >> +-----+-+ +---+---+ +--+----+ +---+---+
> >> smc-id0 | | | |
> >> agent0 | | | |
> >> +-----v--------+---------+-----------+----+
> >> | | | | |
> >> | | | | |
> >> +--------------+---------+-----------+----+
> >> smc-id1 | smc-id2| smc-idX|
> >> agent1 | agent2 | agentX |
> >> | | |
> >> +----v---+ +--v-----+ +--v-----+
> >> | | | | | |
> >> | Dom0 | | Dom1 | | DomX |
> >> | | | | | |
> >> | | | | | |
> >> +--------+ +--------+ +--------+
> >>
> >> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
> >> memory transport for every Agent in the system.
> >>
> >> The SCMI Agent transport channel defined by pair:
> >> - smc-id: SMC/HVC id used for Doorbell
> >> - shmem: shared memory for messages transfer, Xen page aligned,
> >> p2m_mmio_direct_nc.
> >>
> >> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
> >> multi-agent functionality under Xen:
> >> - Xen manegement agent: trusted agents that accesses to the Base Protocol
> >> commands to configure agent specific permissions
> >> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
> >> allowed direct HW access. At least one OSPM VM agent has to be provided
> >> by FW if HW is handled only by Dom0 or Driver Domain.
> >>
> >> The EL3 SCMI FW expected to implement following Base protocol messages:
> >> - BASE_DISCOVER_AGENT
> >> - BASE_RESET_AGENT_CONFIGURATION (optional)
> >> - BASE_SET_DEVICE_PERMISSIONS (optional)
> >>
> >> The SCI SCMI SMC multi-agent driver implements following functionality:
> >> - It's initialized based on the Host DT SCMI node (only one SCMI interface
> >> is supported) which describes Xen management agent SCMI interface.
> >>
> >> scmi_shm_0 : sram@47ff0000 {
> >> compatible = "arm,scmi-shmem";
> >> reg = <0x0 0x47ff0000 0x0 0x1000>;
> >> };
> >> firmware {
> >> scmi: scmi {
> >> compatible = "arm,scmi-smc";
> >> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
> > some extra spaces, it might be a copy/paste error
> +
> >> \#address-cells = < 1>;
> >> \#size-cells = < 0>;
> >> \#access-controller - cells = < 1>;
> >> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
> >>
> >> protocol@X{
> >> };
> >> };
> >> };
> >>
> >> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
> >> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
> >>
> >> chosen {
> >> xen,scmi-secondary-agents = <
> >> 1 0x82000003 &scmi_shm_1
> >> 2 0x82000004 &scmi_shm_2
> >> 3 0x82000005 &scmi_shm_3
> >> 4 0x82000006 &scmi_shm_4>;
> >> }
> >>
> >> /{
> >> scmi_shm_1: sram@47ff1000 {
> >> compatible = "arm,scmi-shmem";
> >> reg = <0x0 0x47ff1000 0x0 0x1000>;
> >> };
> >> scmi_shm_2: sram@47ff2000 {
> >> compatible = "arm,scmi-shmem";
> >> reg = <0x0 0x47ff2000 0x0 0x1000>;
> >> };
> >> scmi_shm_3: sram@47ff3000 {
> >> compatible = "arm,scmi-shmem";
> >> reg = <0x0 0x47ff3000 0x0 0x1000>;
> >> };
> >> }
> >> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
> >> this agent_id.
> >>
> >> Note that Xen is the only one entry in the system which need to know
> >> about SCMI multi-agent support.
> >>
> >> - It implements the SCI subsystem interface required for configuring and
> >> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
> >> SCMI functionality for domain it has to be configured with unique supported
> >> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
> >> [smc-id, shmem] defined for this SCMI Agent_id.
> >> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
> >> -- zero-copy, the guest domain puts SCMI message in shmem;
> >> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
> >> -- the Xen driver catches exception, do checks and synchronously forwards
> >> it to EL3 FW.
> >> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
> >> management agent channel on domain destroy event. This allows to reset
> >> resources used by domain and so implement use-case like domain reboot.
> >>
> >> Dom0 Enable SCMI SMC:
> >> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
> >> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
> >> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
> >> node according to assigned agent_id.
> >>
> >> Guest domains enable SCMI SMC:
> >> - xl.cfg: add configuration option as below
> >>
> >> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
> >>
> >> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
> >> the domain, for example:
> >>
> >> iomem = [
> >> "47ff2,1@22001",
> >> ]
> > Looking at the code and the configuration options, it looks like it is
> > possible to map a scmi-shmem channel at a different address for the
> > guest. It seems like it would work. Is that correct?
> >
> Yes it will. in our case address 22001000 should be the save as
> sram@22001000 in the domain device-tree.
> >> - DT: add SCMI nodes to the Driver domain partial device tree as in the
> >> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
> >>
> >> passthrough {
> >> scmi_shm_0: sram@22001000 {
> >> compatible = "arm,scmi-shmem";
> >> reg = <0x0 0x22001000 0x0 0x1000>;
> >> };
> >>
> >> firmware {
> >> compatible = "simple-bus";
> >> scmi: scmi {
> >> compatible = "arm,scmi-smc";
> >> arm,smc-id = <0x82000004>;
> >> shmem = <&scmi_shm_0>;
> >> ...
> >> }
> >> }
> >> }
> >>
> >> SCMI "4.2.1.1 Device specific access control"
> >>
> >> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
> >> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
> >> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
> >> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
> >> to the Xen SCMI.
> >>
> >> &i2c1 {
> >> access-controllers = <&scmi 0>;
> >> };
> >>
> >> The Dom0 and dom0less domains DT devices will be processed automatically through
> >> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
> >> shell be used:
> >>
> >> dtdev = [
> >> "/soc/i2c@e6508000",
> >> ]
> >>
> >> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
> >>
> >> [1]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
> >> [2]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
> >> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> > Thanks for the long explanation, great work! I am really looking forward
> > to have this feature in the tree soon.
> >
> >
> >> ---
> >>
> >> Changes in v4:
> >> - toolstack comments from Anthony PERARD
> >> - added dom0less support
> >> - added doc for "xen,scmi-secondary-agents"
> >>
> >> docs/man/xl.cfg.5.pod.in | 13 +
> >> docs/misc/arm/device-tree/booting.txt | 60 ++
> >> docs/misc/xen-command-line.pandoc | 9 +
> >> tools/libs/light/libxl_arm.c | 4 +
> >> tools/libs/light/libxl_types.idl | 4 +-
> >> tools/xl/xl_parse.c | 12 +
> >> xen/arch/arm/dom0less-build.c | 11 +
> >> xen/arch/arm/domain_build.c | 3 +-
> >> xen/arch/arm/firmware/Kconfig | 11 +
> >> xen/arch/arm/firmware/Makefile | 1 +
> >> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
> >> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
> >> xen/arch/arm/firmware/scmi-shmem.h | 45 +
> >> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
> >> xen/include/public/arch-arm.h | 3 +
> >> 15 files changed, 1371 insertions(+), 2 deletions(-)
> >> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
> >> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
> >> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
> >> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
> >>
> >> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> >> index 1ccf50b8ea..302c46d8bc 100644
> >> --- a/docs/man/xl.cfg.5.pod.in
> >> +++ b/docs/man/xl.cfg.5.pod.in
> >> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
> >> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
> >> option.
> >>
> >> +=item B<scmi_smc_multiagent>
> >> +
> >> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> >> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
> >> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
> >> +specified for the guest.
> >> +
> >> =back
> >>
> >> +=item B<agent_id=NUMBER>
> >> +
> >> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
> >> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
> >> +existing on a single host must be unique and in the range [1..255].
> >> +
> >> =back
> >>
> >> =back
> >> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> >> index 8943c04173..c8923ab8b2 100644
> >> --- a/docs/misc/arm/device-tree/booting.txt
> >> +++ b/docs/misc/arm/device-tree/booting.txt
> >> @@ -296,6 +296,20 @@ with the following properties:
> >> Should be used together with dom0_scmi_smc_passthrough Xen command line
> >> option.
> >>
> >> + - "scmi_smc_multiagent"
> >> +
> >> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> >> + SMC calls forwarding from domain to the EL3 firmware (like ARM
> >> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
> >> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
> >> + property.
> >> +
> >> +- "xen,sci_agent_id"
> >> +
> >> + Specifies a non-zero ARM SCI agent id for the guest. This option is
> >> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
> >> + the guest. The agent ids of guest must be unique and in the range [1..255].
> >> +
> >> Under the "xen,domain" compatible node, one or more sub-nodes are present
> >> for the DomU kernel and ramdisk.
> >>
> >> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
> >> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
> >> guest physical address space. DomU1 is explicitly defined as the owner domain,
> >> and DomU2 is the borrower domain.
> >> +
> >> +SCMI SMC multi-agent support
> >> +============================
> >> +
> >> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
> >> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
> >> +according to the SCMI compliant EL3 Firmware specification with
> >> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
> >> +the top-level "chosen" node:
> >> +
> >> +- xen,scmi-secondary-agents
> >> +
> >> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
> >> + available for Xen. Each Agent defined as triple consisting of:
> >> + SCMI agent_id,
> >> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
> >> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
> >> +
> >> +As an example:
> >> +
> >> +chosen {
> >> + xen,scmi-secondary-agents = <
> >> + 1 0x82000003 &scmi_shm_1
> >> + 2 0x82000004 &scmi_shm_2
> >> + 3 0x82000005 &scmi_shm_3
> >> + 4 0x82000006 &scmi_shm_4>;
> >> +}
> > NIT: it should be };
> +
> > Looking at scmi_probe, collect_agents, and the following SCMI
> > SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
> > information?
> >
> > It looks like we can discover the agend_ids for every channel, I guess
> > what we need to know is the shmem location for every channel? But the
> > full list of shmem channel is available below thanks to the scmi-shmem
> > nodes.
> >
> > So, we have the list of scmi-shmem anyway, and we can probe the
> > agent_id. The only parameter left is the smc_id/func_id.
> >
> > Or maybe smc_id/func_id can be calculated from agent_id?
> >
> > I am asking mostly because if a user is supposed to add this
> > xen,scmi-secondary-agents property, where are they supposed to find the
> > smc_id/func_id information?
> >
> > It is important that we write down in this document how the user is
> > expected to find out what 1 is 0x82000003 which is scmi_shm_1.
>
> That's a very good question! The issue here is that there are no
>
> explicit requirements defining the relationship between agent_id and
> func_id.
>
>
> For example, in ARM-TF, different implementations can use different
> func_ids.
>
> To provide better flexibility, we decided to separate agent_id from func_id.
>
>
> Currently, the SCMI_BASE_DISCOVER_AGENT calls from the probe are intended to
>
> verify that all registered agents are present and that the configuration
> is correct.
>
> However, I understand that this additional validation could be optional
> to save traffic.
>
>
> To address this, I’m considering adding a configuration option, such as
>
> CONFIG_SCMI_AGENT_VALIDATION, which can be disabled to reduce boot time
> if this
>
> validation is not necessary for certain use cases.
>
>
> Lastly, I’ll be updating the document to include clearer information
> about the
>
> relationship between func_id and agent_id in the upcoming v5.
The key point here is to make it easier for the user. If we can make
agent_id or func_id optional it would make users lives easier.
Alternative, or in addition to this, we should make the docs as clear as
possible so that people can figure it out without having to ask
questions on xen-devel.
> >> +/{
> >> + scmi_shm_1: sram@47ff1000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_2: sram@47ff2000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_3: sram@47ff3000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_3: sram@47ff4000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> >> + };
> > Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
> > Or are under /firmware? Or are they under /chosen?
> >
> > I take they are under the top level node together with scmi_shm_0?
> >
> > Can you please also clarify in the document as well?
> >
> >
> all these nodes are on the top level of the device-tree. But there is no
> specific place for them.
>
> They could be subnodes to some memory-region for example. I will clarify
> this.
>
> >> +}
> >> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> >> index 8e50f6b7c7..bc3c64d6ec 100644
> >> --- a/docs/misc/xen-command-line.pandoc
> >> +++ b/docs/misc/xen-command-line.pandoc
> >> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
> >> SCMI nodes removed from Dom0/hwdom device tree.
> >> (for example, thin Dom0 with Driver domain use-case).
> >>
> >> +### dom0_scmi_agent_id (ARM)
> >> +> `= <integer>`
> >> +
> >> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
> >> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
> >> +The SCMI will be disabled for Dom0 if this option is not specified
> >> +(for example, thin Dom0 or dom0less use-cases).
> >> +The agent ids of domains existing on a single host must be unique.
> >> +
> >> ### dtuart (ARM)
> >> > `= path [:options]`
> >>
> >> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> >> index 28ba9eb787..7712f53cd4 100644
> >> --- a/tools/libs/light/libxl_arm.c
> >> +++ b/tools/libs/light/libxl_arm.c
> >> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> >> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
> >> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> >> break;
> >> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
> >> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> >> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
> >> + break;
> >> default:
> >> LOG(ERROR, "Unknown ARM_SCI type %d",
> >> d_config->b_info.arch_arm.arm_sci.type);
> >> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> >> index aa2190ab5b..11e31ce786 100644
> >> --- a/tools/libs/light/libxl_types.idl
> >> +++ b/tools/libs/light/libxl_types.idl
> >> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
> >>
> >> libxl_arm_sci_type = Enumeration("arm_sci_type", [
> >> (0, "none"),
> >> - (1, "scmi_smc")
> >> + (1, "scmi_smc"),
> >> + (2, "scmi_smc_multiagent")
> >> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
> >>
> >> libxl_arm_sci = Struct("arm_sci", [
> >> ("type", libxl_arm_sci_type),
> >> + ("agent_id", uint8)
> >> ])
> >>
> >> libxl_rdm_reserve = Struct("rdm_reserve", [
> >> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> >> index bd22be9d33..81aa3797e3 100644
> >> --- a/tools/xl/xl_parse.c
> >> +++ b/tools/xl/xl_parse.c
> >> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
> >> }
> >> }
> >>
> >> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
> >> + unsigned long val = parse_ulong(oparg);
> >> +
> >> + if (!val || val > 255) {
> >> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
> >> + val);
> >> + ret = ERROR_INVAL;
> >> + goto parse_error;
> >> + }
> >> + arm_sci->agent_id = val;
> >> + }
> >> +
> >> ptr = strtok(NULL, ",");
> >> }
> >>
> >> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> >> index 0a00f03a25..43d21eb889 100644
> >> --- a/xen/arch/arm/dom0less-build.c
> >> +++ b/xen/arch/arm/dom0less-build.c
> >> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
> >> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> >> else if ( !strcmp(sci_type, "scmi_smc") )
> >> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> >> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
> >> + {
> >> + uint32_t agent_id = 0;
> >> +
> >> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
> >> + !agent_id )
> > shouldn't we check that agent_id <= 255 ?
>
> I see no limitation about max agent_id in DEN0056E document, it's uint32_t.
>
> >> + return -EINVAL;
> >> +
> >> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> >> + d_cfg->arch.arm_sci_agent_id = agent_id;
> >> + }
> >> else
> >> {
> >> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
> >> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> >> index 36d28b52a4..0c9274a2b3 100644
> >> --- a/xen/arch/arm/domain_build.c
> >> +++ b/xen/arch/arm/domain_build.c
> >> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
> >> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
> >> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
> >> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
> >> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
> >> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
> >> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
> >> continue;
> >>
> >> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") ) diff --git a/xen/arch/arm/firmware/Kconfig
> >> b/xen/arch/arm/firmware/Kconfig index 5c5f0880c4..6b051c8ada 100644
> >> --- a/xen/arch/arm/firmware/Kconfig +++
> >> b/xen/arch/arm/firmware/Kconfig @@ -29,6 +29,17 @@ config SCMI_SMC
> >> driver domain. Use with EL3 firmware which supports only single SCMI
> >> OSPM agent. +config SCMI_SMC_MA + bool "Enable ARM SCMI SMC multi-agent driver"
> >> + select ARM_SCI
> >> + help
> >> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
> >> + to EL3 firmware (TF-A) which supports multi-agent feature.
> >> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
> >> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
> >> + allowed platform resources through dedicated SMC/HVC Shared memory based
> >> + transport.
> >> +
> >> endchoice
> >>
> >> endmenu
> >> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
> >> index 71bdefc24a..37927e690e 100644
> >> --- a/xen/arch/arm/firmware/Makefile
> >> +++ b/xen/arch/arm/firmware/Makefile
> >> @@ -1,2 +1,3 @@
> >> obj-$(CONFIG_ARM_SCI) += sci.o
> >> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
> >> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
> >> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
> >> new file mode 100644
> >> index 0000000000..3f4b9c5d6b
> >> --- /dev/null
> >> +++ b/xen/arch/arm/firmware/scmi-proto.h
> >> @@ -0,0 +1,164 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +/*
> >> + * Arm System Control and Management Interface definitions
> >> + * Version 3.0 (DEN0056C)
> >> + *
> >> + * Copyright (c) 2024 EPAM Systems
> >> + */
> >> +
> >> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> >> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> > NIT: ARM_FIRMWARE_SCMI_PROTO_H
> +
> >> +#include <xen/stdint.h>
> >> +
> >> +#define SCMI_SHORT_NAME_MAX_SIZE 16
> >> +
> >> +/* SCMI status codes. See section 4.1.4 */
> >> +#define SCMI_SUCCESS 0
> >> +#define SCMI_NOT_SUPPORTED (-1)
> >> +#define SCMI_INVALID_PARAMETERS (-2)
> >> +#define SCMI_DENIED (-3)
> >> +#define SCMI_NOT_FOUND (-4)
> >> +#define SCMI_OUT_OF_RANGE (-5)
> >> +#define SCMI_BUSY (-6)
> >> +#define SCMI_COMMS_ERROR (-7)
> >> +#define SCMI_GENERIC_ERROR (-8)
> >> +#define SCMI_HARDWARE_ERROR (-9)
> >> +#define SCMI_PROTOCOL_ERROR (-10)
> >> +
> >> +/* Protocol IDs */
> >> +#define SCMI_BASE_PROTOCOL 0x10
> >> +
> >> +/* Base protocol message IDs */
> >> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
> >> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
> >> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
> >> +#define SCMI_BASE_DISCOVER_AGENT 0x7
> >> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
> >> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
> >> +
> >> +typedef struct scmi_msg_header {
> >> + uint8_t id;
> >> + uint8_t type;
> >> + uint8_t protocol;
> >> + uint32_t status;
> >> +} scmi_msg_header_t;
> >> +
> >> +/* Table 2 Message header format */
> >> +#define SCMI_HDR_ID GENMASK(7, 0)
> >> +#define SCMI_HDR_TYPE GENMASK(9, 8)
> >> +#define SCMI_HDR_PROTO GENMASK(17, 10)
> >> +
> >> +#define SCMI_FIELD_GET(_mask, _reg) \
> >> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
> >> +#define SCMI_FIELD_PREP(_mask, _val) \
> >> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
> >> +
> >> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
> >> +{
> >> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
> >> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
> >> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
> >> +}
> >> +
> >> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
> >> +{
> >> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
> >> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
> >> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
> >> +}
> >> +
> >> +static inline int scmi_to_xen_errno(int scmi_status)
> >> +{
> >> + if ( scmi_status == SCMI_SUCCESS )
> >> + return 0;
> >> +
> >> + switch ( scmi_status )
> >> + {
> >> + case SCMI_NOT_SUPPORTED:
> >> + return -EOPNOTSUPP;
> >> + case SCMI_INVALID_PARAMETERS:
> >> + return -EINVAL;
> >> + case SCMI_DENIED:
> >> + return -EACCES;
> >> + case SCMI_NOT_FOUND:
> >> + return -ENOENT;
> >> + case SCMI_OUT_OF_RANGE:
> >> + return -ERANGE;
> >> + case SCMI_BUSY:
> >> + return -EBUSY;
> >> + case SCMI_COMMS_ERROR:
> >> + return -ENOTCONN;
> >> + case SCMI_GENERIC_ERROR:
> >> + return -EIO;
> >> + case SCMI_HARDWARE_ERROR:
> >> + return -ENXIO;
> >> + case SCMI_PROTOCOL_ERROR:
> >> + return -EBADMSG;
> >> + default:
> >> + return -EINVAL;
> >> + }
> >> +}
> >> +
> >> +/* PROTOCOL_VERSION */
> >> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
> >> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
> >> +
> >> +struct scmi_msg_prot_version_p2a {
> >> + uint32_t version;
> >> +} __packed;
> >> +
> >> +/* BASE PROTOCOL_ATTRIBUTES */
> >> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
> >> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
> >> +
> >> +struct scmi_msg_base_attributes_p2a {
> >> + uint32_t attributes;
> >> +} __packed;
> >> +
> >> +/*
> >> + * BASE_DISCOVER_AGENT
> >> + */
> >> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
> >> +
> >> +struct scmi_msg_base_discover_agent_a2p {
> >> + uint32_t agent_id;
> >> +} __packed;
> >> +
> >> +struct scmi_msg_base_discover_agent_p2a {
> >> + uint32_t agent_id;
> >> + char name[SCMI_SHORT_NAME_MAX_SIZE];
> >> +} __packed;
> >> +
> >> +/*
> >> + * BASE_SET_DEVICE_PERMISSIONS
> >> + */
> >> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
> >> +
> >> +struct scmi_msg_base_set_device_permissions_a2p {
> >> + uint32_t agent_id;
> >> + uint32_t device_id;
> >> + uint32_t flags;
> >> +} __packed;
> >> +
> >> +/*
> >> + * BASE_RESET_AGENT_CONFIGURATION
> >> + */
> >> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
> >> +
> >> +struct scmi_msg_base_reset_agent_cfg_a2p {
> >> + uint32_t agent_id;
> >> + uint32_t flags;
> >> +} __packed;
> >> +
> >> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
> >> +
> >> +/*
> >> + * Local variables:
> >> + * mode: C
> >> + * c-file-style: "BSD"
> >> + * c-basic-offset: 4
> >> + * tab-width: 4
> >> + * indent-tabs-mode: nil
> >> + * End:
> >> + */
> >> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
> >> new file mode 100644
> >> index 0000000000..dd613ee0b5
> >> --- /dev/null
> >> +++ b/xen/arch/arm/firmware/scmi-shmem.c
> >> @@ -0,0 +1,173 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +/*
> >> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
> >> + *
> >> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >> + * Copyright (c) 2025 EPAM Systems
> >> + */
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +
> >> +#include <asm/io.h>
> >> +#include <xen/err.h>
> >> +
> >> +#include "scmi-proto.h"
> >> +#include "scmi-shmem.h"
> > This code is written more generically than the description implies. If
> > we only want to make SMC calls to TF-A on EL3 and exchange data with it
> > over shared memory, then I think:
> > - we don't need the __iomem tag, as there is no MMIO
> > - we only need a DMB, not a DSB (readl and writel imply DSB, use only
> > readl_relaxed and writel_relaxed)
> >
> > On the other hand, if we also want to handle the case where the SCMI
> > server could be on a separate co-processor, then what this code is doing
> > is not sufficient because we also need a dcache flush, in addition to
> > the DSB.
> >
> > Bertrand, can you double-check?
> >
> >
> >> +/*
> >> + * Copy data from IO memory space to "real" memory space.
> >> + */
> >> +static void __memcpy_fromio(void *to, const volatile void __iomem *from,
> >> + size_t count)
> >> +{
> >> + while ( count && !IS_ALIGNED((unsigned long)from, 4) )
> >> + {
> >> + *(u8 *)to = readb_relaxed(from);
> >> + from++;
> >> + to++;
> >> + count--;
> >> + }
> >> +
> >> + while ( count >= 4 )
> >> + {
> >> + *(u32 *)to = readl_relaxed(from);
> >> + from += 4;
> >> + to += 4;
> >> + count -= 4;
> >> + }
> >> +
> >> + while ( count )
> >> + {
> >> + *(u8 *)to = readb_relaxed(from);
> >> + from++;
> >> + to++;
> >> + count--;
> >> + }
> >> +}
> >> +
> >> +/*
> >> + * Copy data from "real" memory space to IO memory space.
> >> + */
> >> +static void __memcpy_toio(volatile void __iomem *to, const void *from,
> >> + size_t count)
> >> +{
> >> + while ( count && !IS_ALIGNED((unsigned long)to, 4) )
> >> + {
> >> + writeb_relaxed(*(u8 *)from, to);
> >> + from++;
> >> + to++;
> >> + count--;
> >> + }
> >> +
> >> + while ( count >= 4 )
> >> + {
> >> + writel_relaxed(*(u32 *)from, to);
> >> + from += 4;
> >> + to += 4;
> >> + count -= 4;
> >> + }
> >> +
> >> + while ( count )
> >> + {
> >> + writeb_relaxed(*(u8 *)from, to);
> >> + from++;
> >> + to++;
> >> + count--;
> >> + }
> >> +}
> > I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
> > we use a simple memcpy?
>
> This approach was used because we're trying to access shared memory
> between two independent systems:
>
> Arm-TF and Xen in our case which places some chunks of data to the same
> memory. And, according to the [0]
>
> ```
>
> Some devices (such as framebuffers) would like to use larger transfers than
> 8 bytes at a time. For these devices, the memcpy_toio(),
> memcpy_fromio() and memset_io() functions are
> provided. Do not use memset or memcpy on IO addresses; they are not
> guaranteed to copy data in order.
>
> ```
>
> Also, the same approach was used by Arm team when introducing scmi
> driver to the Linux kernel [1]
>
>
> [0]: https://www.kernel.org/doc/Documentation/driver-api/device-io.rst
>
> [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
Keep in mind that [0] refers specifically to access to MMIO regions. I
assume that the SCMI shared buffers are on normal memory? Regarding [1],
it makes sense if Linux is trying to support shared memory over MMIO.
Looking at one of your replies below, I am guessing the memory buffers
are actually in normal memory but the issue is that TF-A is mapping them
as uncacheable. Is that correct?
In that case, I still don't understand why a simple memcpy would not be
sufficient. Can you check?
If yes, then for now I would just simplify it down to memcpy. When
someone adds support for an SCMI server elsewhere we could look into
adding a more sophisticated memcpy and we can look at the details at
that point in time. Specifically, I am not convinced that memcpy_toio
and memcpy_fromio would work if the SCMI server is on a separate
non-coherent microcontroller.
> >> +static inline int
> >> +shmem_channel_is_free(const volatile struct scmi_shared_mem __iomem *shmem)
> >> +{
> >> + return (readl(&shmem->channel_status) &
> >> + SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE) ? 0 : -EBUSY;
> >> +}
> >> +
> >> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
> >> + scmi_msg_header_t *hdr, void *data, int len)
> >> +{
> >> + int ret;
> >> +
> >> + if ( (len + sizeof(shmem->msg_header)) > SCMI_SHMEM_MAPPED_SIZE )
> >> + {
> >> + printk(XENLOG_ERR "scmi: Wrong size of smc message. Data is invalid\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + ret = shmem_channel_is_free(shmem);
> >> + if ( ret )
> >> + return ret;
> >> +
> >> + writel_relaxed(0x0, &shmem->channel_status);
> >> + /* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
> >> + writel_relaxed(0x0, &shmem->flags);
> >> + writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
> >> + writel(pack_scmi_header(hdr), &shmem->msg_header);
> >> +
> >> + if ( len > 0 && data )
> >> + __memcpy_toio(shmem->msg_payload, data, len);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
> >> + scmi_msg_header_t *hdr, void *data, int len)
> >> +{
> >> + int recv_len;
> >> + int ret;
> >> + int pad = sizeof(hdr->status);
> >> +
> >> + if ( len >= SCMI_SHMEM_MAPPED_SIZE - sizeof(shmem) )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: Wrong size of input smc message. Data may be invalid\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + ret = shmem_channel_is_free(shmem);
> >> + if ( ret )
> >> + return ret;
> >> +
> >> + recv_len = readl(&shmem->length) - sizeof(shmem->msg_header);
> >> +
> >> + if ( recv_len < 0 )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: Wrong size of smc message. Data may be invalid\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + unpack_scmi_header(readl(&shmem->msg_header), hdr);
> >> +
> >> + hdr->status = readl(&shmem->msg_payload);
> >> + recv_len = recv_len > pad ? recv_len - pad : 0;
> >> +
> >> + ret = scmi_to_xen_errno(hdr->status);
> >> + if ( ret )
> >> + {
> >> + printk(XENLOG_DEBUG "scmi: Error received: %d\n", ret);
> >> + return ret;
> >> + }
> >> +
> >> + if ( recv_len > len )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: Not enough buffer for message %d, expecting %d\n",
> >> + recv_len, len);
> >> + return -EINVAL;
> >> + }
> >> +
> >> + if ( recv_len > 0 )
> >> + __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +/*
> >> + * Local variables:
> >> + * mode: C
> >> + * c-file-style: "BSD"
> >> + * c-basic-offset: 4
> >> + * tab-width: 4
> >> + * indent-tabs-mode: nil
> >> + * End:
> >> + */
> >> diff --git a/xen/arch/arm/firmware/scmi-shmem.h b/xen/arch/arm/firmware/scmi-shmem.h
> >> new file mode 100644
> >> index 0000000000..2f8e23ff76
> >> --- /dev/null
> >> +++ b/xen/arch/arm/firmware/scmi-shmem.h
> >> @@ -0,0 +1,45 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +/*
> >> + * Arm System Control and Management Interface definitions
> >> + * Version 3.0 (DEN0056C)
> >> + * Shared Memory based Transport
> >> + *
> >> + * Copyright (c) 2024 EPAM Systems
> >> + */
> >> +
> >> +#ifndef XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
> >> +#define XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
> > NIT: ARM_FIRMWARE_SCMI_SHMEM_H
> >
> +
> >> +#include <xen/stdint.h>
> >> +
> >> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE BIT(0, UL)
> >> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_ERROR BIT(1, UL)
> >> +
> >> +struct scmi_shared_mem {
> >> + uint32_t reserved;
> >> + uint32_t channel_status;
> >> + uint32_t reserved1[2];
> >> + uint32_t flags;
> >> + uint32_t length;
> >> + uint32_t msg_header;
> >> + uint8_t msg_payload[];
> >> +};
> >> +
> >> +#define SCMI_SHMEM_MAPPED_SIZE PAGE_SIZE
> >> +
> >> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
> >> + scmi_msg_header_t *hdr, void *data, int len);
> >> +
> >> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
> >> + scmi_msg_header_t *hdr, void *data, int len);
> >> +#endif /* XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_ */
> >> +
> >> +/*
> >> + * Local variables:
> >> + * mode: C
> >> + * c-file-style: "BSD"
> >> + * c-basic-offset: 4
> >> + * tab-width: 4
> >> + * indent-tabs-mode: nil
> >> + * End:
> >> + */
> >> diff --git a/xen/arch/arm/firmware/scmi-smc-multiagent.c b/xen/arch/arm/firmware/scmi-smc-multiagent.c
> >> new file mode 100644
> >> index 0000000000..e023bca3a1
> >> --- /dev/null
> >> +++ b/xen/arch/arm/firmware/scmi-smc-multiagent.c
> >> @@ -0,0 +1,860 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +/*
> >> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
> >> + *
> >> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >> + * Copyright (c) 2025 EPAM Systems
> >> + */
> >> +
> >> +#include <xen/acpi.h>
> >> +
> >> +#include <xen/device_tree.h>
> >> +#include <xen/init.h>
> >> +#include <xen/iocap.h>
> >> +#include <xen/err.h>
> >> +#include <xen/libfdt/libfdt.h>
> >> +#include <xen/param.h>
> >> +#include <xen/sched.h>
> >> +#include <xen/vmap.h>
> >> +
> >> +#include <asm/firmware/sci.h>
> >> +#include <asm/smccc.h>
> >> +
> >> +#include "scmi-proto.h"
> >> +#include "scmi-shmem.h"
> >> +
> >> +#define SCMI_AGENT_ID_INVALID 0xFF
> >> +
> >> +static uint8_t __initdata opt_dom0_scmi_agent_id = SCMI_AGENT_ID_INVALID;
> >> +integer_param("dom0_scmi_agent_id", opt_dom0_scmi_agent_id);
> >> +
> >> +#define SCMI_SECONDARY_AGENTS "xen,scmi-secondary-agents"
> >> +
> >> +#define HYP_CHANNEL 0x0
> >> +
> >> +struct scmi_channel {
> >> + uint32_t agent_id;
> >> + uint32_t func_id;
> >> + domid_t domain_id;
> >> + uint64_t paddr;
> >> + uint64_t len;
> >> + struct scmi_shared_mem __iomem *shmem;
> >> + spinlock_t lock;
> >> + struct list_head list;
> >> +};
> >> +
> >> +struct scmi_data {
> >> + struct list_head channel_list;
> >> + spinlock_t channel_list_lock;
> >> + uint32_t func_id;
> >> + bool initialized;
> >> + uint32_t shmem_phandle;
> >> + struct dt_device_node *dt_dev;
> >> +};
> >> +
> >> +static struct scmi_data scmi_data;
> >> +
> >> +static int send_smc_message(struct scmi_channel *chan_info,
> >> + scmi_msg_header_t *hdr, void *data, int len)
> >> +{
> >> + struct arm_smccc_res resp;
> >> + int ret;
> >> +
> >> + ret = shmem_put_message(chan_info->shmem, hdr, data, len);
> >> + if ( ret )
> >> + return ret;
> >> +
> >> + arm_smccc_1_1_smc(chan_info->func_id, 0, 0, 0, 0, 0, 0, 0, &resp);
> >> +
> >> + if ( resp.a0 )
> >> + return -EOPNOTSUPP;
> > Why if repo.a0 != 0 then we assume -EOPNOTSUPP? Is this part of the SCMI
> > specification?
>
> Please see 7.1 of [1]. The following codes are supported :
>
> 0 for success
>
> -1 as not_supported
>
> -2 as not_required
>
> -3 as invalid_parameter.
>
> For all non-zero results we assume that this call is not supported by
> system or has different revision
>
> which we also doesn't support.
For -1 and -2 make sense. I am less sure about converting -3 into
-EOPNOTSUPP. But I'll trust your judgment.
> [1] DEN0028
> https://documentation-service.arm.com/static/5f8edaeff86e16515cdbe4c6
> >> + return 0;
> >> +}
> >> +
> >> +static int do_smc_xfer(struct scmi_channel *chan_info, scmi_msg_header_t *hdr,
> >> + void *tx_data, int tx_size, void *rx_data, int rx_size)
> >> +{
> >> + int ret = 0;
> >> +
> >> + ASSERT(chan_info && chan_info->shmem);
> >> +
> >> + if ( !hdr )
> >> + return -EINVAL;
> >> +
> >> + spin_lock(&chan_info->lock);
> >> +
> >> + printk(XENLOG_DEBUG
> >> + "scmi: agent_id = %d msg_id = %x type = %d, proto = %x\n",
> >> + chan_info->agent_id, hdr->id, hdr->type, hdr->protocol);
> >> +
> >> + ret = send_smc_message(chan_info, hdr, tx_data, tx_size);
> >> + if ( ret )
> >> + goto clean;
> >> +
> >> + ret = shmem_get_response(chan_info->shmem, hdr, rx_data, rx_size);
> >> +
> >> +clean:
> >> + printk(XENLOG_DEBUG
> >> + "scmi: get smc response agent_id = %d msg_id = %x proto = %x res=%d\n",
> >> + chan_info->agent_id, hdr->id, hdr->protocol, ret);
> >> +
> >> + spin_unlock(&chan_info->lock);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static struct scmi_channel *get_channel_by_id(uint32_t agent_id)
> >> +{
> >> + struct scmi_channel *curr;
> >> + bool found = false;
> >> +
> >> + spin_lock(&scmi_data.channel_list_lock);
> >> + list_for_each_entry(curr, &scmi_data.channel_list, list)
> >> + {
> >> + if ( curr->agent_id == agent_id )
> >> + {
> >> + found = true;
> >> + break;
> >> + }
> >> + }
> >> +
> >> + spin_unlock(&scmi_data.channel_list_lock);
> >> + if ( found )
> >> + return curr;
> >> +
> >> + return NULL;
> >> +}
> >> +
> >> +static struct scmi_channel *acquire_scmi_channel(struct domain *d,
> >> + uint32_t agent_id)
> >> +{
> >> + struct scmi_channel *curr;
> >> + struct scmi_channel *ret = ERR_PTR(-ENOENT);
> >> +
> >> + spin_lock(&scmi_data.channel_list_lock);
> >> + list_for_each_entry(curr, &scmi_data.channel_list, list)
> >> + {
> >> + if ( curr->agent_id == agent_id )
> >> + {
> >> + if ( curr->domain_id != DOMID_INVALID )
> >> + {
> >> + ret = ERR_PTR(-EEXIST);
> >> + break;
> >> + }
> >> +
> >> + curr->domain_id = d->domain_id;
> >> + ret = curr;
> >> + break;
> >> + }
> >> + }
> >> +
> >> + spin_unlock(&scmi_data.channel_list_lock);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static void relinquish_scmi_channel(struct scmi_channel *channel)
> >> +{
> >> + ASSERT(channel != NULL);
> >> +
> >> + spin_lock(&scmi_data.channel_list_lock);
> >> + channel->domain_id = DOMID_INVALID;
> >> + spin_unlock(&scmi_data.channel_list_lock);
> >> +}
> >> +
> >> +static int map_channel_memory(struct scmi_channel *channel)
> >> +{
> >> + ASSERT(channel && channel->paddr);
> >> + channel->shmem = ioremap_nocache(channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
> > ioremap is for MMIO, if these shared memory channels are on DDR, then it
> > would not be the right call. Are the "arm,scmi-shmem" address ranges
> > part of the memory node ranges? Or are they completely separate?
> >
> > Also, why nocache? Wouldn't we want ioremap_cache?
> >
> Currently, they are separate nodes but could potentially be part of the
> "memory-region".
>
> From what I see in the Linux kernel, devm_ioremap is being used to map
> the memory.
>
> Could there be something more to this approach?
>
> [0]:https://git.iliana.fyi/linux/patch/?id=1dc6558062dadfabd2fb3bd885fa6e92ec7196f2
I am guessing that TF-A is mapping memory as uncacheable so we should do
the same in Xen.
> >> + if ( !channel->shmem )
> >> + return -ENOMEM;
> >> +
> >> + channel->shmem->channel_status = SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE;
> >> + printk(XENLOG_DEBUG "scmi: Got shmem %lx after vmap %p\n", channel->paddr,
> >> + channel->shmem);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static void unmap_channel_memory(struct scmi_channel *channel)
> >> +{
> >> + ASSERT(channel && channel->shmem);
> >> + iounmap(channel->shmem);
> >> + channel->shmem = NULL;
> >> +}
> >> +
> >> +static struct scmi_channel *smc_create_channel(uint32_t agent_id,
> >> + uint32_t func_id, uint64_t addr)
> >> +{
> >> + struct scmi_channel *channel;
> >> +
> >> + channel = get_channel_by_id(agent_id);
> >> + if ( channel )
> >> + return ERR_PTR(EEXIST);
> >> +
> >> + channel = xmalloc(struct scmi_channel);
> >> + if ( !channel )
> >> + return ERR_PTR(ENOMEM);
> >> +
> >> + spin_lock_init(&channel->lock);
> >> + channel->agent_id = agent_id;
> >> + channel->func_id = func_id;
> >> + channel->domain_id = DOMID_INVALID;
> >> + channel->shmem = NULL;
> >> + channel->paddr = addr;
> >> + list_add_tail(&channel->list, &scmi_data.channel_list);
> >> + return channel;
> >> +}
> >> +
> >> +static void free_channel_list(void)
> >> +{
> >> + struct scmi_channel *curr, *_curr;
> >> +
> >> + list_for_each_entry_safe(curr, _curr, &scmi_data.channel_list, list)
> >> + {
> >> + list_del(&curr->list);
> >> + xfree(curr);
> >> + }
> >> +}
> >> +
> >> +static int __init
> >> +scmi_dt_read_hyp_channel_addr(struct dt_device_node *scmi_node, u64 *addr,
> >> + u64 *size)
> >> +{
> >> + struct dt_device_node *shmem_node;
> >> + const __be32 *prop;
> >> +
> >> + prop = dt_get_property(scmi_node, "shmem", NULL);
> >> + if ( !prop )
> >> + return -EINVAL;
> >> +
> >> + shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
> >> + if ( IS_ERR_OR_NULL(shmem_node) )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: Device tree error, can't parse reserved memory %ld\n",
> >> + PTR_ERR(shmem_node));
> >> + return PTR_ERR(shmem_node);
> >> + }
> >> +
> >> + return dt_device_get_address(shmem_node, 0, addr, size);
> >> +}
> >> +
> >> +/*
> >> + * Handle Dom0 SCMI specific DT nodes
> >> + *
> >> + * Make a decision on copying SCMI specific nodes into Dom0 device tree.
> >> + * For SCMI multi-agent case:
> >> + * - shmem nodes will not be copied and generated instead if SCMI
> >> + * is enabled for Dom0
> >> + * - scmi node will be copied if SCMI is enabled for Dom0
> >> + */
> >> +static bool scmi_dt_handle_node(struct domain *d, struct dt_device_node *node)
> >> +{
> >> + static const struct dt_device_match skip_matches[] __initconst = {
> >> + DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
> >> + { /* sentinel */ },
> >> + };
> >> + static const struct dt_device_match scmi_matches[] __initconst = {
> >> + DT_MATCH_PATH("/firmware/scmi"),
> >> + { /* sentinel */ },
> >> + };
> >> +
> >> + if ( !scmi_data.initialized )
> >> + return false;
> >> +
> >> + /* always drop shmem */
> >> + if ( dt_match_node(skip_matches, node) )
> >> + {
> >> + dt_dprintk(" Skip scmi shmem\n");
> >> + return true;
> >> + }
> >> +
> >> + /* drop scmi if not enabled */
> >> + if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
> >> + {
> >> + dt_dprintk(" Skip scmi node\n");
> >> + return true;
> >> + }
> >> +
> >> + return false;
> >> +}
> >> +
> >> +/*
> >> + * Finalize Dom0 SCMI specific DT nodes
> >> + *
> >> + * if SCMI is enabled for Dom0:
> >> + * - generate shmem node
> >> + * - map SCMI shmem MMIO into Dom0
> >> + */
> >> +static int scmi_dt_finalize(struct domain *d, void *fdt)
> >> +{
> >> + __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
> >> + struct scmi_channel *channel;
> >> + int nodeoffset;
> >> + __be32 *cells;
> >> + __be32 val;
> >> + char buf[64];
> >> + int res, rc;
> >> +
> >> + if ( !sci_domain_is_enabled(d) )
> >> + return 0;
> >> +
> >> + channel = d->arch.sci_data;
> >> +
> >> + /*
> >> + * Replace "arm,smc-id" with proper value assigned for Dom0 SCMI channel
> >> + */
> >> + nodeoffset = fdt_node_offset_by_compatible(fdt, -1, "arm,scmi-smc");
> >> + if ( nodeoffset < 0 )
> >> + return -ENODEV;
> >> +
> >> + cells = (__be32 *)&val;
> >> + dt_set_cell(&cells, 1, channel->func_id);
> >> + res = fdt_setprop_inplace(fdt, nodeoffset, "arm,smc-id", &val, sizeof(val));
> >> + if ( res )
> >> + return -EINVAL;
> >> +
> > Are you sure it is worth to go through all this trouble to modify FDT in
> > place when we could simply generate the DT node from scratch like we do
> > for example for the GIC? This seems to be more error prone as well. Is
> > generating it from scratch is really difficult? If it is difficult then OK.
> >
> In the last patch [0] of the series, there is a proposal to separate
>
> the Xen privileged agent from the Dom0 agent. This eliminates the
>
> need to modify the Xen Device Tree Blob (DTB), which is a positive
> improvement
>
> since the Dom0 agent node is already present in the device tree.
>
> However, the issue with SCMI node generation lies in the need to handle
>
> the list of protocols that are supported across the system.
>
> If we want to generate the SCMI node for Dom0, we need to:
>
> Copy these protocols from a centralized or predefined source.
> Set the correct phandle for each subnode within the SCMI node, ensuring
>
> accurate representation and functionality.
> This extra step of managing the protocols and phandles adds
>
> complexity but is necessary for ensuring proper support for Dom0.
>
> [0]
> https://lists.xenproject.org/archives/html/xen-devel/2025-05/msg01041.html
I was only commenting that rather than trying to modify the DT in place
we could create the node for Dom0 from scratch (artificially), based on
host DT information as required (fetching data from the host DT as
required and copying it to the Dom0 DT).
> >> + /*
> >> + * All SCMI shmem nodes should be removed from Dom0 DT at this point, so
> >> + * the shmem node for Dom0 need to be generated from SCMI channel assigned
> >> + * to Dom0.
> >> + * The original SCMI shmem node from platform DT is used by Xen SCMI driver
> >> + * itself as privileged channel (agent_id=0) to manage other SCMI
> >> + * agents (domains).
> >> + */
> >> + snprintf(buf, sizeof(buf), "scmi-shmem@%lx", channel->paddr);
> >> +
> >> + res = fdt_begin_node(fdt, buf);
> >> + if ( res )
> >> + return res;
> >> +
> >> + res = fdt_property_string(fdt, "compatible", "arm,scmi-shmem");
> >> + if ( res )
> >> + return res;
> >> +
> >> + cells = ®[0];
> >> +
> >> + dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> >> + channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
> >> +
> >> + res = fdt_property(fdt, "reg", reg, sizeof(reg));
> >> + if ( res )
> >> + return res;
> >> +
> >> + res = fdt_property_cell(fdt, "phandle", scmi_data.shmem_phandle);
> >> + if ( res )
> >> + return res;
> >> +
> >> + res = fdt_end_node(fdt);
> >> + if ( res )
> >> + return res;
> >> +
> >> + /*
> >> + * Map SCMI shmem into Dom0 here as shmem nodes are excluded from
> >> + * generic Dom0 DT processing
> >> + */
> >> + res = iomem_permit_access(d, paddr_to_pfn(channel->paddr),
> >> + paddr_to_pfn(channel->paddr +
> >> + SCMI_SHMEM_MAPPED_SIZE - 1));
> >> + if ( res )
> >> + return res;
> >> +
> >> + res = map_regions_p2mt(d, gaddr_to_gfn(channel->paddr),
> >> + PFN_UP(SCMI_SHMEM_MAPPED_SIZE),
> >> + maddr_to_mfn(channel->paddr), p2m_mmio_direct_nc);
> >> + if ( res )
> >> + {
> >> + rc = iomem_deny_access(d, paddr_to_pfn(channel->paddr),
> >> + paddr_to_pfn(channel->paddr +
> >> + SCMI_SHMEM_MAPPED_SIZE - 1));
> >> + if ( rc )
> >> + printk(XENLOG_ERR "scmi: Unable to deny iomem access , err = %d\n",
> >> + rc);
> >> + }
> >> +
> >> + return res;
> >> +}
> >> +
> >> +static int scmi_assign_device(uint32_t agent_id, uint32_t device_id,
> >> + uint32_t flags)
> >> +{
> >> + struct scmi_msg_base_set_device_permissions_a2p tx;
> >> + struct scmi_channel *channel;
> >> + scmi_msg_header_t hdr;
> >> + int ret;
> >> +
> >> + channel = get_channel_by_id(HYP_CHANNEL);
> >> + if ( !channel )
> >> + return -EINVAL;
> >> +
> >> + hdr.id = SCMI_BASE_SET_DEVICE_PERMISSIONS;
> >> + hdr.type = 0;
> >> + hdr.protocol = SCMI_BASE_PROTOCOL;
> >> +
> >> + tx.agent_id = agent_id;
> >> + tx.device_id = device_id;
> >> + tx.flags = flags;
> >> +
> >> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
> >> + if ( ret == -EOPNOTSUPP )
> >> + return 0;
> > Is it actually OK to pretend that everything worked if the return is
> > -EOPNOTSUPP? I mean that in this case can we assume that the device is
> > actually assigned anyway? Wouldn't follow up SCMI operations on this
> > device fail?
> >
> I think you right. Will fix in v5
> >> + return ret;
> >> +}
> >> +
> >> +static int scmi_dt_assign_device(struct domain *d,
> >> + struct dt_phandle_args *ac_spec)
> >> +{
> >> + struct scmi_channel *agent_channel;
> >> + uint32_t scmi_device_id = ac_spec->args[0];
> >> + int ret;
> >> +
> >> + if ( !d->arch.sci_data )
> >> + return 0;
> >> +
> >> + /* The access-controllers is specified for DT dev, but it's not a SCMI */
> >> + if ( ac_spec->np != scmi_data.dt_dev )
> >> + return 0;
> > I wonder if this should be an error
> >
> We don’t expect the DT node access-controller to rely solely on SCMI.
> Therefore,
>
> if a DT node has an access-controller but it’s not associated with SCMI,
> we should simply
>
> ignore that node without raising an error.
>
>
> That’s my understanding of the situation.
I understand that, but should scmi_dt_assign_device be called in such as
a case? I didn't go back and look at the original code in writing this
comment.
> >> + agent_channel = d->arch.sci_data;
> >> +
> >> + spin_lock(&agent_channel->lock);
> >> +
> >> + ret = scmi_assign_device(agent_channel->agent_id, scmi_device_id,
> >> + SCMI_BASE_DEVICE_ACCESS_ALLOW);
> >> + if ( ret )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: could not assign dev for %pd agent:%d dev_id:%u (%d)",
> >> + d, agent_channel->agent_id, scmi_device_id, ret);
> >> + }
> >> +
> >> + spin_unlock(&agent_channel->lock);
> >> + return ret;
> >> +}
> >> +
> >> +static __init int collect_agents(struct dt_device_node *scmi_node)
> >> +{
> >> + const struct dt_device_node *chosen_node;
> >> + const __be32 *prop;
> >> + uint32_t len, i;
> >> +
> >> + chosen_node = dt_find_node_by_path("/chosen");
> >> + if ( !chosen_node )
> >> + {
> >> + printk(XENLOG_ERR "scmi: chosen node not found\n");
> >> + return -ENOENT;
> >> + }
> >> +
> >> + prop = dt_get_property(chosen_node, SCMI_SECONDARY_AGENTS, &len);
> >> + if ( !prop )
> >> + {
> >> + printk(XENLOG_WARNING "scmi: No %s property found\n",
> >> + SCMI_SECONDARY_AGENTS);
> >> + return -ENODEV;
> >> + }
> >> +
> >> + if ( len % (3 * sizeof(uint32_t)) )
> >> + {
> >> + printk(XENLOG_ERR "scmi: Invalid length of %s property: %d\n",
> >> + SCMI_SECONDARY_AGENTS, len);
> >> + return -EINVAL;
> >> + }
> >> +
> >> + for ( i = 0; i < len / (3 * sizeof(uint32_t)); i++ )
> >> + {
> >> + uint32_t agent_id = be32_to_cpu(*prop++);
> >> + uint32_t smc_id = be32_to_cpu(*prop++);
> >> + uint32_t shmem_phandle = be32_to_cpu(*prop++);
> >> + struct dt_device_node *node = dt_find_node_by_phandle(shmem_phandle);
> >> + u64 addr, size;
> >> + int ret;
> >> +
> >> + if ( !node )
> >> + {
> >> + printk(XENLOG_ERR "scmi: Could not find shmem node for agent %d\n",
> >> + agent_id);
> >> + return -EINVAL;
> >> + }
> >> +
> >> + ret = dt_device_get_address(node, 0, &addr, &size);
> >> + if ( ret )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: Could not read shmem address for agent %d: %d",
> >> + agent_id, ret);
> >> + return ret;
> >> + }
> >> +
> >> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
> >> + {
> >> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + ret = PTR_RET(smc_create_channel(agent_id, smc_id, addr));
> >> + if ( ret )
> >> + {
> >> + printk(XENLOG_ERR "scmi: Could not create channel for agent %d: %d",
> >> + agent_id, ret);
> >> + return ret;
> >> + }
> >> +
> >> + printk(XENLOG_DEBUG "scmi: Agent %d SMC %X addr %lx\n", agent_id,
> >> + smc_id, addr);
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int scmi_domain_init(struct domain *d,
> >> + struct xen_domctl_createdomain *config)
> >> +{
> >> + struct scmi_channel *channel;
> >> + int ret;
> >> +
> >> + if ( !scmi_data.initialized )
> >> + return 0;
> >> +
> >> + /*
> >> + * Special case for Dom0 - the SCMI support is enabled basing on
> >> + * "dom0_sci_agent_id" Xen command line parameter
> >> + */
> >> + if ( is_hardware_domain(d) )
> >> + {
> >> + if ( opt_dom0_scmi_agent_id != SCMI_AGENT_ID_INVALID )
> >> + {
> >> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> >> + config->arch.arm_sci_agent_id = opt_dom0_scmi_agent_id;
> >> + }
> >> + else
> >> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> >> + }
> >> +
> >> + if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE )
> >> + return 0;
> >> +
> >> + channel = acquire_scmi_channel(d, config->arch.arm_sci_agent_id);
> >> + if ( IS_ERR(channel) )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: Failed to acquire SCMI channel for agent_id %u: %ld\n",
> >> + config->arch.arm_sci_agent_id, PTR_ERR(channel));
> >> + return PTR_ERR(channel);
> >> + }
> >> +
> >> + printk(XENLOG_INFO
> >> + "scmi: Acquire channel id = 0x%x, domain_id = %d paddr = 0x%lx\n",
> >> + channel->agent_id, channel->domain_id, channel->paddr);
> >> +
> >> + /*
> >> + * Dom0 (if present) needs to have an access to the guest memory range
> >> + * to satisfy iomem_access_permitted() check in XEN_DOMCTL_iomem_permission
> >> + * domctl.
> > Ideally this should not be needed but I understand we don't have an
> > easy solution, I think we can go ahead with this for now.
> >
> >> + */
> >> + if ( hardware_domain && !is_hardware_domain(d) )
> >> + {
> >> + ret = iomem_permit_access(hardware_domain, paddr_to_pfn(channel->paddr),
> >> + paddr_to_pfn(channel->paddr + PAGE_SIZE - 1));
> >> + if ( ret )
> >> + goto error;
> >> + }
> >> +
> >> + d->arch.sci_data = channel;
> >> + d->arch.sci_enabled = true;
> >> +
> >> + return 0;
> >> +
> >> +error:
> >> + relinquish_scmi_channel(channel);
> >> + return ret;
> >> +}
> >> +
> >> +int scmi_domain_sanitise_config(struct xen_domctl_createdomain *config)
> >> +{
> >> + if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
> >> + config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA )
> >> + {
> >> + dprintk(XENLOG_INFO, "scmi: Unsupported ARM_SCI type\n");
> >> + return -EINVAL;
> >> + }
> >> + else if ( config->arch.arm_sci_type ==
> >> + XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA &&
> >> + config->arch.arm_sci_agent_id == 0 )
> >> + {
> >> + dprintk(XENLOG_INFO,
> >> + "scmi: A zero ARM SCMI agent_id is not supported\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int scmi_relinquish_resources(struct domain *d)
> >> +{
> >> + int ret;
> >> + struct scmi_channel *channel, *agent_channel;
> >> + scmi_msg_header_t hdr;
> >> + struct scmi_msg_base_reset_agent_cfg_a2p tx;
> >> +
> >> + if ( !d->arch.sci_data )
> >> + return 0;
> >> +
> >> + agent_channel = d->arch.sci_data;
> >> +
> >> + spin_lock(&agent_channel->lock);
> >> + tx.agent_id = agent_channel->agent_id;
> >> + spin_unlock(&agent_channel->lock);
> >> +
> >> + channel = get_channel_by_id(HYP_CHANNEL);
> >> + if ( !channel )
> >> + {
> >> + printk(XENLOG_ERR
> >> + "scmi: Unable to get Hypervisor scmi channel for domain %d\n",
> >> + d->domain_id);
> >> + return -EINVAL;
> >> + }
> >> +
> >> + hdr.id = SCMI_BASE_RESET_AGENT_CONFIGURATION;
> >> + hdr.type = 0;
> >> + hdr.protocol = SCMI_BASE_PROTOCOL;
> >> +
> >> + tx.flags = 0;
> >> +
> >> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
> >> + if ( ret == -EOPNOTSUPP )
> >> + return 0;
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static void scmi_domain_destroy(struct domain *d)
> >> +{
> >> + struct scmi_channel *channel;
> >> +
> >> + if ( !d->arch.sci_data )
> >> + return;
> >> +
> >> + channel = d->arch.sci_data;
> >> + spin_lock(&channel->lock);
> >> +
> >> + relinquish_scmi_channel(channel);
> >> + printk(XENLOG_DEBUG "scmi: Free domain %d\n", d->domain_id);
> >> +
> >> + d->arch.sci_data = NULL;
> >> + d->arch.sci_enabled = true;
> >> +
> >> + spin_unlock(&channel->lock);
> >> +}
> >> +
> >> +static bool scmi_handle_call(struct cpu_user_regs *regs)
> >> +{
> >> + uint32_t fid = (uint32_t)get_user_reg(regs, 0);
> >> + struct scmi_channel *agent_channel;
> >> + struct domain *d = current->domain;
> >> + struct arm_smccc_res resp;
> >> + bool res = false;
> >> +
> >> + if ( !sci_domain_is_enabled(d) )
> >> + return false;
> >> +
> >> + agent_channel = d->arch.sci_data;
> >> + spin_lock(&agent_channel->lock);
> >> +
> >> + if ( agent_channel->func_id != fid )
> >> + {
> >> + res = false;
> >> + goto unlock;
> >> + }
> >> +
> >> + arm_smccc_1_1_smc(fid,
> >> + get_user_reg(regs, 1),
> >> + get_user_reg(regs, 2),
> >> + get_user_reg(regs, 3),
> >> + get_user_reg(regs, 4),
> >> + get_user_reg(regs, 5),
> >> + get_user_reg(regs, 6),
> >> + get_user_reg(regs, 7),
> >> + &resp);
> >> +
> >> + set_user_reg(regs, 0, resp.a0);
> >> + set_user_reg(regs, 1, resp.a1);
> >> + set_user_reg(regs, 2, resp.a2);
> >> + set_user_reg(regs, 3, resp.a3);
> >> + res = true;
> >> +unlock:
> >> + spin_unlock(&agent_channel->lock);
> >> +
> >> + return res;
> >> +}
> >> +
> >> +static const struct sci_mediator_ops scmi_ops = {
> >> + .domain_init = scmi_domain_init,
> >> + .domain_destroy = scmi_domain_destroy,
> >> + .relinquish_resources = scmi_relinquish_resources,
> >> + .handle_call = scmi_handle_call,
> >> + .dom0_dt_handle_node = scmi_dt_handle_node,
> >> + .dom0_dt_finalize = scmi_dt_finalize,
> >> + .domain_sanitise_config = scmi_domain_sanitise_config,
> >> + .assign_dt_device = scmi_dt_assign_device,
> >> +};
> >> +
> >> +static int __init scmi_check_smccc_ver(void)
> >> +{
> >> + if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
> >> + {
> >> + printk(XENLOG_WARNING
> >> + "scmi: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
> >> + return -ENOSYS;
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static __init int scmi_probe(struct dt_device_node *scmi_node, const void *data)
> >> +{
> >> + u64 addr, size;
> >> + int ret, i;
> >> + struct scmi_channel *channel, *agent_channel;
> >> + int n_agents;
> >> + scmi_msg_header_t hdr;
> >> + struct scmi_msg_base_attributes_p2a rx;
> >> +
> >> + ASSERT(scmi_node != NULL);
> >> +
> >> + INIT_LIST_HEAD(&scmi_data.channel_list);
> >> + spin_lock_init(&scmi_data.channel_list_lock);
> >> +
> >> + if ( !acpi_disabled )
> >> + {
> >> + printk(XENLOG_WARNING "scmi: is not supported when using ACPI\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + ret = scmi_check_smccc_ver();
> >> + if ( ret )
> >> + return ret;
> >> +
> >> + if ( !dt_property_read_u32(scmi_node, "arm,smc-id", &scmi_data.func_id) )
> >> + {
> >> + printk(XENLOG_ERR "scmi: unable to read smc-id from DT\n");
> >> + return -ENOENT;
> >> + }
> >> +
> >> + /* save shmem phandle and re-use it fro Dom0 DT shmem node */
> >> + if ( !dt_property_read_u32(scmi_node, "shmem", &scmi_data.shmem_phandle) )
> >> + {
> >> + printk(XENLOG_ERR "scmi: unable to read shmem phandle from DT\n");
> >> + return -ENOENT;
> >> + }
> >> +
> >> + ret = scmi_dt_read_hyp_channel_addr(scmi_node, &addr, &size);
> >> + if ( IS_ERR_VALUE(ret) )
> >> + return -ENOENT;
> >> +
> >> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
> >> + {
> >> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + scmi_data.dt_dev = scmi_node;
> >> +
> >> + channel = smc_create_channel(HYP_CHANNEL, scmi_data.func_id, addr);
> >> + if ( IS_ERR(channel) )
> >> + goto out;
> >> +
> >> + ret = map_channel_memory(channel);
> >> + if ( ret )
> >> + goto out;
> >> +
> >> + channel->domain_id = DOMID_XEN;
> >> +
> >> + hdr.id = SCMI_BASE_PROTOCOL_ATTIBUTES;
> >> + hdr.type = 0;
> >> + hdr.protocol = SCMI_BASE_PROTOCOL;
> >> +
> >> + ret = do_smc_xfer(channel, &hdr, NULL, 0, &rx, sizeof(rx));
> >> + if ( ret )
> >> + goto error;
> >> +
> >> + n_agents = SCMI_FIELD_GET(SCMI_BASE_ATTR_NUM_AGENT, rx.attributes);
> >> + printk(XENLOG_DEBUG "scmi: Got agent count %d\n", n_agents);
> >> +
> >> + ret = collect_agents(scmi_node);
> >> + if ( ret )
> >> + goto error;
> >> +
> >> + i = 1;
> >> +
> >> + list_for_each_entry(agent_channel, &scmi_data.channel_list, list)
> >> + {
> >> + struct scmi_msg_base_discover_agent_p2a da_rx;
> >> + struct scmi_msg_base_discover_agent_a2p da_tx;
> >> +
> >> + ret = map_channel_memory(agent_channel);
> >> + if ( ret )
> >> + goto error;
> >> +
> >> + hdr.id = SCMI_BASE_DISCOVER_AGENT;
> >> + hdr.type = 0;
> >> + hdr.protocol = SCMI_BASE_PROTOCOL;
> >> +
> >> + da_tx.agent_id = agent_channel->agent_id;
> >> +
> >> + ret = do_smc_xfer(agent_channel, &hdr, &da_tx, sizeof(da_tx), &da_rx,
> >> + sizeof(da_rx));
> >> + if ( agent_channel->domain_id != DOMID_XEN )
> >> + unmap_channel_memory(agent_channel);
> >> + if ( ret )
> >> + goto error;
> >> +
> >> + printk(XENLOG_DEBUG "id=0x%x name=%s\n", da_rx.agent_id, da_rx.name);
> >> +
> >> + agent_channel->agent_id = da_rx.agent_id;
> > It is OK to set agent_channel->agent_id to the value provided by the
> > SCMI server, but if we are also taking the agent_channel->agent_id value
> > from the user via device tree, shouldn't we throw an error if there is a
> > mismatch?
> >
> > Or even better: can we avoid taking the value via device tree to make it
> > easier to configure?
> >
> I plan to drop the last line, as there’s no need to overwrite agent_id.
>
> However, we cannot avoid obtaining agent_id from the device tree because,
>
> according to section 4.2.2.9 [0], an agent can only be discovered by its
> agent_id in
>
> a privileged domain. Alternatively, each agent can obtain its own ID by
> providing
>
> 0xFFFFFFFF as agent_id and using its own channel.
That would be great! Anything to reduce the number of required (not
optional) configuration options.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-12 12:10 ` Grygorii Strashko
@ 2025-06-17 23:38 ` Stefano Stabellini
2025-06-18 7:22 ` Julien Grall
2025-06-19 16:15 ` Oleksii Moisieiev
0 siblings, 2 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-17 23:38 UTC (permalink / raw)
To: Grygorii Strashko
Cc: Bertrand Marquis, Stefano Stabellini, Oleksii Moisieiev,
xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Jan Beulich, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk
On Thu, 12 Jun 2025, Grygorii Strashko wrote:
> On 02.06.25 10:17, Bertrand Marquis wrote:
> > Hi Stefano and Oleksii,
> >
> > > On 23 May 2025, at 22:06, Stefano Stabellini <sstabellini@kernel.org>
> > > wrote:
> > >
> > > One question for Bertrand below
> > >
> > >
> > > On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> > > > This patch introduces SCI driver to support for ARM EL3 Trusted
> > > > Firmware-A
> > > > (TF-A) which provides SCMI interface with multi-agnet support, as shown
> > > > below.
> > > >
> > > > +-----------------------------------------+
> > > > | |
> > > > | EL3 TF-A SCMI |
> > > > +-------+--+-------+--+-------+--+-------++
> > > > |shmem0 | |shmem1 | |shmem2 | |shmemX |
> > > > +-----+-+ +---+---+ +--+----+ +---+---+
> > > > smc-id0 | | | |
> > > > agent0 | | | |
> > > > +-----v--------+---------+-----------+----+
> > > > | | | | |
> > > > | | | | |
> > > > +--------------+---------+-----------+----+
> > > > smc-id1 | smc-id2| smc-idX|
> > > > agent1 | agent2 | agentX |
> > > > | | |
> > > > +----v---+ +--v-----+ +--v-----+
> > > > | | | | | |
> > > > | Dom0 | | Dom1 | | DomX |
> > > > | | | | | |
> > > > | | | | | |
> > > > +--------+ +--------+ +--------+
> > > >
> > > > The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC
> > > > shared
> > > > memory transport for every Agent in the system.
> > > >
> > > > The SCMI Agent transport channel defined by pair:
> > > > - smc-id: SMC/HVC id used for Doorbell
> > > > - shmem: shared memory for messages transfer, Xen page aligned,
> > > > p2m_mmio_direct_nc.
> > > >
> > > > The follwoing SCMI Agents expected to be defined by SCMI FW to enable
> > > > SCMI
> > > > multi-agent functionality under Xen:
> > > > - Xen manegement agent: trusted agents that accesses to the Base
> > > > Protocol
> > > > commands to configure agent specific permissions
> > > > - OSPM VM agents: non-trusted agent, one for each Guest domain which is
> > > > allowed direct HW access. At least one OSPM VM agent has to be
> > > > provided
> > > > by FW if HW is handled only by Dom0 or Driver Domain.
> > > >
> > > > The EL3 SCMI FW expected to implement following Base protocol messages:
> > > > - BASE_DISCOVER_AGENT
> > > > - BASE_RESET_AGENT_CONFIGURATION (optional)
> > > > - BASE_SET_DEVICE_PERMISSIONS (optional)
> > > >
> > > > The SCI SCMI SMC multi-agent driver implements following functionality:
> > > > - It's initialized based on the Host DT SCMI node (only one SCMI
> > > > interface
> > > > is supported) which describes Xen management agent SCMI interface.
> > > >
> > > > scmi_shm_0 : sram@47ff0000 {
> > > > compatible = "arm,scmi-shmem";
> > > > reg = <0x0 0x47ff0000 0x0 0x1000>;
> > > > };
> > > > firmware {
> > > > scmi: scmi {
> > > > compatible = "arm,scmi-smc";
> > > > arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
> > >
> > > some extra spaces, it might be a copy/paste error
> > >
> > >
> > > > \#address-cells = < 1>;
> > > > \#size-cells = < 0>;
> > > > \#access-controller - cells = < 1>;
> > > > shmem = <&scmi_shm_0>; // Xen manegement agent shmem
> > > >
> > > > protocol@X{
> > > > };
> > > > };
> > > > };
> > > >
> > > > - It obtains Xen specific SCMI Agent's configuration from the Host DT,
> > > > probes Agents and build SCMI Agents list; The Agents configuration is
> > > > taken from:
> > > >
> > > > chosen {
> > > > xen,scmi-secondary-agents = <
> > > > 1 0x82000003 &scmi_shm_1
> > > > 2 0x82000004 &scmi_shm_2
> > > > 3 0x82000005 &scmi_shm_3
> > > > 4 0x82000006 &scmi_shm_4>;
> > > > }
> > > >
> > > > /{
> > > > scmi_shm_1: sram@47ff1000 {
> > > > compatible = "arm,scmi-shmem";
> > > > reg = <0x0 0x47ff1000 0x0 0x1000>;
> > > > };
> > > > scmi_shm_2: sram@47ff2000 {
> > > > compatible = "arm,scmi-shmem";
> > > > reg = <0x0 0x47ff2000 0x0 0x1000>;
> > > > };
> > > > scmi_shm_3: sram@47ff3000 {
> > > > compatible = "arm,scmi-shmem";
> > > > reg = <0x0 0x47ff3000 0x0 0x1000>;
> > > > };
> > > > }
> > > > where first item is "agent_id", second - "arm,smc-id", and third -
> > > > "arm,scmi-shmem" for
> > > > this agent_id.
> > > >
> > > > Note that Xen is the only one entry in the system which need to know
> > > > about SCMI multi-agent support.
> > > >
> > > > - It implements the SCI subsystem interface required for configuring and
> > > > enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
> > > > SCMI functionality for domain it has to be configured with unique
> > > > supported
> > > > SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
> > > > [smc-id, shmem] defined for this SCMI Agent_id.
> > > > - Once Xen domain is configured it can communicate with EL3 SCMI FW:
> > > > -- zero-copy, the guest domain puts SCMI message in shmem;
> > > > -- the guest triggers SMC/HVC exception with smc-id (doorbell);
> > > > -- the Xen driver catches exception, do checks and synchronously
> > > > forwards
> > > > it to EL3 FW.
> > > > - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
> > > > management agent channel on domain destroy event. This allows to reset
> > > > resources used by domain and so implement use-case like domain reboot.
> > > >
> > > > Dom0 Enable SCMI SMC:
> > > > - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not
> > > > provided
> > > > SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0
> > > > DT.
> > > > The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up
> > > > shmem
> > > > node according to assigned agent_id.
> > > >
> > > > Guest domains enable SCMI SMC:
> > > > - xl.cfg: add configuration option as below
> > > >
> > > > arm_sci = "type=scmi_smc_multiagent,agent_id=2"
> > > >
> > > > - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond
> > > > assigned agent_id for
> > > > the domain, for example:
> > > >
> > > > iomem = [
> > > > "47ff2,1@22001",
> > > > ]
> > >
> > > Looking at the code and the configuration options, it looks like it is
> > > possible to map a scmi-shmem channel at a different address for the
> > > guest. It seems like it would work. Is that correct?
> > >
> > >
> > > > - DT: add SCMI nodes to the Driver domain partial device tree as in the
> > > > below example. The "arm,smc-id" should correspond assigned agent_id for
> > > > the domain:
> > > >
> > > > passthrough {
> > > > scmi_shm_0: sram@22001000 {
> > > > compatible = "arm,scmi-shmem";
> > > > reg = <0x0 0x22001000 0x0 0x1000>;
> > > > };
> > > >
> > > > firmware {
> > > > compatible = "simple-bus";
> > > > scmi: scmi {
> > > > compatible = "arm,scmi-smc";
> > > > arm,smc-id = <0x82000004>;
> > > > shmem = <&scmi_shm_0>;
> > > > ...
> > > > }
> > > > }
> > > > }
> > > >
> > > > SCMI "4.2.1.1 Device specific access control"
> > > >
> > > > The XEN SCI SCMI SMC multi-agent driver performs "access-controller"
> > > > provider function
> > > > in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access
> > > > control" and provides the
> > > > BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an
> > > > agents have access to.
> > > > The DT SCMI node should "#access-controller-cells=<1>" property and DT
> > > > devices should be bound
> > > > to the Xen SCMI.
> > > >
> > > > &i2c1 {
> > > > access-controllers = <&scmi 0>;
> > > > };
> > > >
> > > > The Dom0 and dom0less domains DT devices will be processed automatically
> > > > through
> > > > sci_assign_dt_device() call, but to assign SCMI devices from toolstack
> > > > the xl.cfg:"dtdev" property
> > > > shell be used:
> > > >
> > > > dtdev = [
> > > > "/soc/i2c@e6508000",
> > > > ]
> > > >
> > > > xl.cfg:dtdev will contain all nodes which are under SCMI management (not
> > > > only those which are behind IOMMU).
> > > >
> > > > [1]
> > > > https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
> > > > [2]
> > > > https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
> > > > Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> > > > Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> > >
> > > Thanks for the long explanation, great work! I am really looking forward
> > > to have this feature in the tree soon.
> > >
> > >
> > > > ---
> > > >
> > > > Changes in v4:
> > > > - toolstack comments from Anthony PERARD
> > > > - added dom0less support
> > > > - added doc for "xen,scmi-secondary-agents"
> > > >
> > > > docs/man/xl.cfg.5.pod.in | 13 +
> > > > docs/misc/arm/device-tree/booting.txt | 60 ++
> > > > docs/misc/xen-command-line.pandoc | 9 +
> > > > tools/libs/light/libxl_arm.c | 4 +
> > > > tools/libs/light/libxl_types.idl | 4 +-
> > > > tools/xl/xl_parse.c | 12 +
> > > > xen/arch/arm/dom0less-build.c | 11 +
> > > > xen/arch/arm/domain_build.c | 3 +-
> > > > xen/arch/arm/firmware/Kconfig | 11 +
> > > > xen/arch/arm/firmware/Makefile | 1 +
> > > > xen/arch/arm/firmware/scmi-proto.h | 164 ++++
> > > > xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
> > > > xen/arch/arm/firmware/scmi-shmem.h | 45 +
> > > > xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
> > > > xen/include/public/arch-arm.h | 3 +
> > > > 15 files changed, 1371 insertions(+), 2 deletions(-)
> > > > create mode 100644 xen/arch/arm/firmware/scmi-proto.h
> > > > create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
> > > > create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
> > > > create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
> > > >
> > > > diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> > > > index 1ccf50b8ea..302c46d8bc 100644
> > > > --- a/docs/man/xl.cfg.5.pod.in
> > > > +++ b/docs/man/xl.cfg.5.pod.in
> > > > @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
> > > > Should be used together with B<dom0_scmi_smc_passthrough> Xen command
> > > > line
> > > > option.
> > > >
> > > > +=item B<scmi_smc_multiagent>
> > > > +
> > > > +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI
> > > > over
> > > > +SMC calls forwarding from domain to the EL3 firmware (like Trusted
> > > > Firmware-A)
> > > > +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
> > > > +specified for the guest.
> > > > +
> > > > =back
> > > >
> > > > +=item B<agent_id=NUMBER>
> > > > +
> > > > +Specifies a non-zero ARM SCI agent id for the guest. This option is
> > > > mandatory
> > > > +if the SCMI SMC support is enabled for the guest. The agent ids of
> > > > domains
> > > > +existing on a single host must be unique and in the range [1..255].
> > > > +
> > > > =back
> > > >
> > > > =back
> > > > diff --git a/docs/misc/arm/device-tree/booting.txt
> > > > b/docs/misc/arm/device-tree/booting.txt
> > > > index 8943c04173..c8923ab8b2 100644
> > > > --- a/docs/misc/arm/device-tree/booting.txt
> > > > +++ b/docs/misc/arm/device-tree/booting.txt
> > > > @@ -296,6 +296,20 @@ with the following properties:
> > > > Should be used together with dom0_scmi_smc_passthrough Xen command
> > > > line
> > > > option.
> > > >
> > > > + - "scmi_smc_multiagent"
> > > > +
> > > > + Enables ARM SCMI SMC multi-agent support for the guest by enabling
> > > > SCMI over
> > > > + SMC calls forwarding from domain to the EL3 firmware (like ARM
> > > > + Trusted Firmware-A) with a multi SCMI OSPM agent support.
> > > > + The SCMI agent_id should be specified for the guest with
> > > > "xen,sci_agent_id"
> > > > + property.
> > > > +
> > > > +- "xen,sci_agent_id"
> > > > +
> > > > + Specifies a non-zero ARM SCI agent id for the guest. This option is
> > > > + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled
> > > > for
> > > > + the guest. The agent ids of guest must be unique and in the range
> > > > [1..255].
> > > > +
> > > > Under the "xen,domain" compatible node, one or more sub-nodes are
> > > > present
> > > > for the DomU kernel and ramdisk.
> > > >
> > > > @@ -764,3 +778,49 @@ The automatically allocated static shared memory
> > > > will get mapped at
> > > > 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in
> > > > DomU2
> > > > guest physical address space. DomU1 is explicitly defined as the owner
> > > > domain,
> > > > and DomU2 is the borrower domain.
> > > > +
> > > > +SCMI SMC multi-agent support
> > > > +============================
> > > > +
> > > > +For enabling the ARM SCMI SMC multi-agent support (enabled by
> > > > CONFIG_SCMI_SMC_MA)
> > > > +the Xen specific SCMI Agent's configuration shell be provided in the
> > > > Host DT
> > > > +according to the SCMI compliant EL3 Firmware specification with
> > > > +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
> > > > +the top-level "chosen" node:
> > > > +
> > > > +- xen,scmi-secondary-agents
> > > > +
> > > > + Defines a set of SCMI agents configuration supported by SCMI EL3 FW
> > > > and
> > > > + available for Xen. Each Agent defined as triple consisting of:
> > > > + SCMI agent_id,
> > > > + SMC/HVC function_id assigned for the agent transport
> > > > ("arm,smc-id"),
> > > > + phandle to SCMI SHM assigned for the agent transport
> > > > ("arm,scmi-shmem").
> > > > +
> > > > +As an example:
> > > > +
> > > > +chosen {
> > > > + xen,scmi-secondary-agents = <
> > > > + 1 0x82000003 &scmi_shm_1
> > > > + 2 0x82000004 &scmi_shm_2
> > > > + 3 0x82000005 &scmi_shm_3
> > > > + 4 0x82000006 &scmi_shm_4>;
> > > > +}
> > >
> > > NIT: it should be };
> > >
> > > Looking at scmi_probe, collect_agents, and the following SCMI
> > > SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
> > > information?
> > >
> > > It looks like we can discover the agend_ids for every channel, I guess
> > > what we need to know is the shmem location for every channel? But the
> > > full list of shmem channel is available below thanks to the scmi-shmem
> > > nodes.
> > >
> > > So, we have the list of scmi-shmem anyway, and we can probe the
> > > agent_id. The only parameter left is the smc_id/func_id.
> > >
> > > Or maybe smc_id/func_id can be calculated from agent_id?
> > >
> > > I am asking mostly because if a user is supposed to add this
> > > xen,scmi-secondary-agents property, where are they supposed to find the
> > > smc_id/func_id information?
> > >
> > > It is important that we write down in this document how the user is
> > > expected to find out what 1 is 0x82000003 which is scmi_shm_1.
> > >
> > >
> > > > +/{
> > > > + scmi_shm_1: sram@47ff1000 {
> > > > + compatible = "arm,scmi-shmem";
> > > > + reg = <0x0 0x47ff1000 0x0 0x1000>;
> > > > + };
> > > > + scmi_shm_2: sram@47ff2000 {
> > > > + compatible = "arm,scmi-shmem";
> > > > + reg = <0x0 0x47ff2000 0x0 0x1000>;
> > > > + };
> > > > + scmi_shm_3: sram@47ff3000 {
> > > > + compatible = "arm,scmi-shmem";
> > > > + reg = <0x0 0x47ff3000 0x0 0x1000>;
> > > > + };
> > > > + scmi_shm_3: sram@47ff4000 {
> > > > + compatible = "arm,scmi-shmem";
> > > > + reg = <0x0 0x47ff4000 0x0 0x1000>;
> > > > + };
> > >
> > > Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
> > > Or are under /firmware? Or are they under /chosen?
> > >
> > > I take they are under the top level node together with scmi_shm_0?
> > >
> > > Can you please also clarify in the document as well?
> > >
> > >
> > > > +}
> > > > diff --git a/docs/misc/xen-command-line.pandoc
> > > > b/docs/misc/xen-command-line.pandoc
> > > > index 8e50f6b7c7..bc3c64d6ec 100644
> > > > --- a/docs/misc/xen-command-line.pandoc
> > > > +++ b/docs/misc/xen-command-line.pandoc
> > > > @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be
> > > > disabled for Dom0/hwdom and
> > > > SCMI nodes removed from Dom0/hwdom device tree.
> > > > (for example, thin Dom0 with Driver domain use-case).
> > > >
> > > > +### dom0_scmi_agent_id (ARM)
> > > > +> `= <integer>`
> > > > +
> > > > +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and
> > > > allows to
> > > > +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI
> > > > agent id.
> > > > +The SCMI will be disabled for Dom0 if this option is not specified
> > > > +(for example, thin Dom0 or dom0less use-cases).
> > > > +The agent ids of domains existing on a single host must be unique.
> > > > +
> > > > ### dtuart (ARM)
> > > > > `= path [:options]`
> > > >
> > > > diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> > > > index 28ba9eb787..7712f53cd4 100644
> > > > --- a/tools/libs/light/libxl_arm.c
> > > > +++ b/tools/libs/light/libxl_arm.c
> > > > @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc
> > > > *gc,
> > > > case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
> > > > config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> > > > break;
> > > > + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
> > > > + config->arch.arm_sci_type =
> > > > XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> > > > + config->arch.arm_sci_agent_id =
> > > > d_config->b_info.arch_arm.arm_sci.agent_id;
> > > > + break;
> > > > default:
> > > > LOG(ERROR, "Unknown ARM_SCI type %d",
> > > > d_config->b_info.arch_arm.arm_sci.type);
> > > > diff --git a/tools/libs/light/libxl_types.idl
> > > > b/tools/libs/light/libxl_types.idl
> > > > index aa2190ab5b..11e31ce786 100644
> > > > --- a/tools/libs/light/libxl_types.idl
> > > > +++ b/tools/libs/light/libxl_types.idl
> > > > @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
> > > >
> > > > libxl_arm_sci_type = Enumeration("arm_sci_type", [
> > > > (0, "none"),
> > > > - (1, "scmi_smc")
> > > > + (1, "scmi_smc"),
> > > > + (2, "scmi_smc_multiagent")
> > > > ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
> > > >
> > > > libxl_arm_sci = Struct("arm_sci", [
> > > > ("type", libxl_arm_sci_type),
> > > > + ("agent_id", uint8)
> > > > ])
> > > >
> > > > libxl_rdm_reserve = Struct("rdm_reserve", [
> > > > diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> > > > index bd22be9d33..81aa3797e3 100644
> > > > --- a/tools/xl/xl_parse.c
> > > > +++ b/tools/xl/xl_parse.c
> > > > @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg,
> > > > libxl_arm_sci *arm_sci,
> > > > }
> > > > }
> > > >
> > > > + if (MATCH_OPTION("agent_id", ptr, oparg)) {
> > > > + unsigned long val = parse_ulong(oparg);
> > > > +
> > > > + if (!val || val > 255) {
> > > > + fprintf(stderr, "An invalid ARM_SCI agent_id specified
> > > > (%lu). Valid range [1..255]\n",
> > > > + val);
> > > > + ret = ERROR_INVAL;
> > > > + goto parse_error;
> > > > + }
> > > > + arm_sci->agent_id = val;
> > > > + }
> > > > +
> > > > ptr = strtok(NULL, ",");
> > > > }
> > > >
> > > > diff --git a/xen/arch/arm/dom0less-build.c
> > > > b/xen/arch/arm/dom0less-build.c
> > > > index 0a00f03a25..43d21eb889 100644
> > > > --- a/xen/arch/arm/dom0less-build.c
> > > > +++ b/xen/arch/arm/dom0less-build.c
> > > > @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node
> > > > *node,
> > > > d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> > > > else if ( !strcmp(sci_type, "scmi_smc") )
> > > > d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> > > > + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
> > > > + {
> > > > + uint32_t agent_id = 0;
> > > > +
> > > > + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id)
> > > > ||
> > > > + !agent_id )
> > >
> > > shouldn't we check that agent_id <= 255 ?
> > >
> > >
> > > > + return -EINVAL;
> > > > +
> > > > + d_cfg->arch.arm_sci_type =
> > > > XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> > > > + d_cfg->arch.arm_sci_agent_id = agent_id;
> > > > + }
> > > > else
> > > > {
> > > > printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain
> > > > %s\n",
> > > > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > > > index 36d28b52a4..0c9274a2b3 100644
> > > > --- a/xen/arch/arm/domain_build.c
> > > > +++ b/xen/arch/arm/domain_build.c
> > > > @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d,
> > > > struct kernel_info *kinfo,
> > > > dt_property_name_is_equal(prop,
> > > > "linux,uefi-mmap-start") ||
> > > > dt_property_name_is_equal(prop,
> > > > "linux,uefi-mmap-size") ||
> > > > dt_property_name_is_equal(prop,
> > > > "linux,uefi-mmap-desc-size") ||
> > > > - dt_property_name_is_equal(prop,
> > > > "linux,uefi-mmap-desc-ver"))
> > > > + dt_property_name_is_equal(prop,
> > > > "linux,uefi-mmap-desc-ver") ||
> > > > + dt_property_name_is_equal(prop,
> > > > "xen,scmi-secondary-agents") )
> > > > continue;
> > > >
> > > > if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") )
> > > > diff --git a/xen/arch/arm/firmware/Kconfig
> > > > b/xen/arch/arm/firmware/Kconfig
> > > > index 5c5f0880c4..6b051c8ada 100644
> > > > --- a/xen/arch/arm/firmware/Kconfig
> > > > +++ b/xen/arch/arm/firmware/Kconfig
> > > > @@ -29,6 +29,17 @@ config SCMI_SMC
> > > > driver domain.
> > > > Use with EL3 firmware which supports only single SCMI OSPM agent.
> > > >
> > > > +config SCMI_SMC_MA
> > > > + bool "Enable ARM SCMI SMC multi-agent driver"
> > > > + select ARM_SCI
> > > > + help
> > > > + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from
> > > > Domains
> > > > + to EL3 firmware (TF-A) which supports multi-agent feature.
> > > > + This feature allows to enable SCMI per Domain using unique SCMI
> > > > agent_id,
> > > > + so Domain is identified by EL3 firmware as an SCMI Agent and can
> > > > access
> > > > + allowed platform resources through dedicated SMC/HVC Shared memory
> > > > based
> > > > + transport.
> > > > +
> > > > endchoice
> > > >
> > > > endmenu
> > > > diff --git a/xen/arch/arm/firmware/Makefile
> > > > b/xen/arch/arm/firmware/Makefile
> > > > index 71bdefc24a..37927e690e 100644
> > > > --- a/xen/arch/arm/firmware/Makefile
> > > > +++ b/xen/arch/arm/firmware/Makefile
> > > > @@ -1,2 +1,3 @@
> > > > obj-$(CONFIG_ARM_SCI) += sci.o
> > > > obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
> > > > +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
> > > > diff --git a/xen/arch/arm/firmware/scmi-proto.h
> > > > b/xen/arch/arm/firmware/scmi-proto.h
> > > > new file mode 100644
> > > > index 0000000000..3f4b9c5d6b
> > > > --- /dev/null
> > > > +++ b/xen/arch/arm/firmware/scmi-proto.h
> > > > @@ -0,0 +1,164 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > +/*
> > > > + * Arm System Control and Management Interface definitions
> > > > + * Version 3.0 (DEN0056C)
> > > > + *
> > > > + * Copyright (c) 2024 EPAM Systems
> > > > + */
> > > > +
> > > > +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> > > > +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> > >
> > > NIT: ARM_FIRMWARE_SCMI_PROTO_H
> > >
> > >
> > > > +#include <xen/stdint.h>
> > > > +
> > > > +#define SCMI_SHORT_NAME_MAX_SIZE 16
> > > > +
> > > > +/* SCMI status codes. See section 4.1.4 */
> > > > +#define SCMI_SUCCESS 0
> > > > +#define SCMI_NOT_SUPPORTED (-1)
> > > > +#define SCMI_INVALID_PARAMETERS (-2)
> > > > +#define SCMI_DENIED (-3)
> > > > +#define SCMI_NOT_FOUND (-4)
> > > > +#define SCMI_OUT_OF_RANGE (-5)
> > > > +#define SCMI_BUSY (-6)
> > > > +#define SCMI_COMMS_ERROR (-7)
> > > > +#define SCMI_GENERIC_ERROR (-8)
> > > > +#define SCMI_HARDWARE_ERROR (-9)
> > > > +#define SCMI_PROTOCOL_ERROR (-10)
> > > > +
> > > > +/* Protocol IDs */
> > > > +#define SCMI_BASE_PROTOCOL 0x10
> > > > +
> > > > +/* Base protocol message IDs */
> > > > +#define SCMI_BASE_PROTOCOL_VERSION 0x0
> > > > +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
> > > > +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
> > > > +#define SCMI_BASE_DISCOVER_AGENT 0x7
> > > > +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
> > > > +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
> > > > +
> > > > +typedef struct scmi_msg_header {
> > > > + uint8_t id;
> > > > + uint8_t type;
> > > > + uint8_t protocol;
> > > > + uint32_t status;
> > > > +} scmi_msg_header_t;
> > > > +
> > > > +/* Table 2 Message header format */
> > > > +#define SCMI_HDR_ID GENMASK(7, 0)
> > > > +#define SCMI_HDR_TYPE GENMASK(9, 8)
> > > > +#define SCMI_HDR_PROTO GENMASK(17, 10)
> > > > +
> > > > +#define SCMI_FIELD_GET(_mask, _reg)
> > > > \
> > > > + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
> > > > +#define SCMI_FIELD_PREP(_mask, _val)
> > > > \
> > > > + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
> > > > +
> > > > +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
> > > > +{
> > > > + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
> > > > + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
> > > > + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
> > > > +}
> > > > +
> > > > +static inline void unpack_scmi_header(uint32_t msg_hdr,
> > > > scmi_msg_header_t *hdr)
> > > > +{
> > > > + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
> > > > + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
> > > > + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
> > > > +}
> > > > +
> > > > +static inline int scmi_to_xen_errno(int scmi_status)
> > > > +{
> > > > + if ( scmi_status == SCMI_SUCCESS )
> > > > + return 0;
> > > > +
> > > > + switch ( scmi_status )
> > > > + {
> > > > + case SCMI_NOT_SUPPORTED:
> > > > + return -EOPNOTSUPP;
> > > > + case SCMI_INVALID_PARAMETERS:
> > > > + return -EINVAL;
> > > > + case SCMI_DENIED:
> > > > + return -EACCES;
> > > > + case SCMI_NOT_FOUND:
> > > > + return -ENOENT;
> > > > + case SCMI_OUT_OF_RANGE:
> > > > + return -ERANGE;
> > > > + case SCMI_BUSY:
> > > > + return -EBUSY;
> > > > + case SCMI_COMMS_ERROR:
> > > > + return -ENOTCONN;
> > > > + case SCMI_GENERIC_ERROR:
> > > > + return -EIO;
> > > > + case SCMI_HARDWARE_ERROR:
> > > > + return -ENXIO;
> > > > + case SCMI_PROTOCOL_ERROR:
> > > > + return -EBADMSG;
> > > > + default:
> > > > + return -EINVAL;
> > > > + }
> > > > +}
> > > > +
> > > > +/* PROTOCOL_VERSION */
> > > > +#define SCMI_VERSION_MINOR GENMASK(15, 0)
> > > > +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
> > > > +
> > > > +struct scmi_msg_prot_version_p2a {
> > > > + uint32_t version;
> > > > +} __packed;
> > > > +
> > > > +/* BASE PROTOCOL_ATTRIBUTES */
> > > > +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
> > > > +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
> > > > +
> > > > +struct scmi_msg_base_attributes_p2a {
> > > > + uint32_t attributes;
> > > > +} __packed;
> > > > +
> > > > +/*
> > > > + * BASE_DISCOVER_AGENT
> > > > + */
> > > > +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
> > > > +
> > > > +struct scmi_msg_base_discover_agent_a2p {
> > > > + uint32_t agent_id;
> > > > +} __packed;
> > > > +
> > > > +struct scmi_msg_base_discover_agent_p2a {
> > > > + uint32_t agent_id;
> > > > + char name[SCMI_SHORT_NAME_MAX_SIZE];
> > > > +} __packed;
> > > > +
> > > > +/*
> > > > + * BASE_SET_DEVICE_PERMISSIONS
> > > > + */
> > > > +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
> > > > +
> > > > +struct scmi_msg_base_set_device_permissions_a2p {
> > > > + uint32_t agent_id;
> > > > + uint32_t device_id;
> > > > + uint32_t flags;
> > > > +} __packed;
> > > > +
> > > > +/*
> > > > + * BASE_RESET_AGENT_CONFIGURATION
> > > > + */
> > > > +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
> > > > +
> > > > +struct scmi_msg_base_reset_agent_cfg_a2p {
> > > > + uint32_t agent_id;
> > > > + uint32_t flags;
> > > > +} __packed;
> > > > +
> > > > +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
> > > > +
> > > > +/*
> > > > + * Local variables:
> > > > + * mode: C
> > > > + * c-file-style: "BSD"
> > > > + * c-basic-offset: 4
> > > > + * tab-width: 4
> > > > + * indent-tabs-mode: nil
> > > > + * End:
> > > > + */
> > > > diff --git a/xen/arch/arm/firmware/scmi-shmem.c
> > > > b/xen/arch/arm/firmware/scmi-shmem.c
> > > > new file mode 100644
> > > > index 0000000000..dd613ee0b5
> > > > --- /dev/null
> > > > +++ b/xen/arch/arm/firmware/scmi-shmem.c
> > > > @@ -0,0 +1,173 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > +/*
> > > > + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
> > > > + *
> > > > + * Oleksii Moisieiev <oleksii_moisieiev@epam.com>
> > > > + * Copyright (c) 2025 EPAM Systems
> > > > + */
> > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > +
> > > > +#include <asm/io.h>
> > > > +#include <xen/err.h>
> > > > +
> > > > +#include "scmi-proto.h"
> > > > +#include "scmi-shmem.h"
> > >
> > > This code is written more generically than the description implies. If
> > > we only want to make SMC calls to TF-A on EL3 and exchange data with it
> > > over shared memory, then I think:
> > > - we don't need the __iomem tag, as there is no MMIO
> >
> > agree
> >
> > > - we only need a DMB, not a DSB (readl and writel imply DSB, use only
> > > readl_relaxed and writel_relaxed)
> >
> > agree
> >
> > >
> > > On the other hand, if we also want to handle the case where the SCMI
> > > server could be on a separate co-processor, then what this code is doing
> > > is not sufficient because we also need a dcache flush, in addition to
> > > the DSB.
> > >
> > > Bertrand, can you double-check?
> >
> > If we want to handle a case where the memory is accessible to a coprocessor
> > but there is no cache coherency, we need to flush the dcache definitely.
> >
> > Seeing the amount of data here, I do agree with Stefano that it would be a
> > good
> > idea to make the provision to flush the data cache in all cases. Even if the
> > data
> > is accessed by a secure partition or the firmware coherently, flushing in
> > all cases
> > would have very limited performance impact here.
> >
> > There is the other solution to have some kind of parameter to say if the
> > accessor
> > has coherent cache access but I do not think the performance impact here
> > would
> > justify such a complexity.
> >
> The SCMI shmem expected to be mapped as MT_NON_CACHEABLE in all cases.
Is that part of the spec?
> The Linux does devm_ioremap() -> ioremap() ->
> (ARM64) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
>
> There is also note in docs:
> "+- shmem: shared memory for messages transfer, **Xen page aligned** with
> mapping``p2m_mmio_direct_nc``."
>
> In the case of SCP - the SCMI shmem can be actually be in SRAM.
>
> So, are you sure cache manipulations are required here?
No, if the memory is mapped as uncacheable everywhere then the cache
manipulations are not needed. However, we probably still need a dsb.
I understand now why they decided to use __memcpy_fromio in Linux: it is
not MMIO but they needed a memcpy followed by DSB, so they decided to
reuse the existing MMIO functions although the buffer is not MMIO.
In Xen I would prefer we just used memcpy followed by DSB. The opinion
of other maintainers is very welcome.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-17 23:22 ` Stefano Stabellini
@ 2025-06-17 23:56 ` Stefano Stabellini
2025-06-19 16:15 ` Oleksii Moisieiev
1 sibling, 0 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-17 23:56 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Oleksii Moisieiev, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Julien Grall, Michal Orzel, Roger Pau Monné,
Volodymyr Babchuk, Grygorii Strashko
[-- Attachment #1: Type: text/plain, Size: 35577 bytes --]
On Tue, 17 Jun 2025, Stefano Stabellini wrote:
> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> > Hi Stefano,
> >
> > I'm very sorry for a long silence. Please see my answers below:
> >
> >
> > On 23/05/2025 23:06, Stefano Stabellini wrote:
> > > One question for Bertrand below
> > >
> > >
> > > On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> > >> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
> > >> (TF-A) which provides SCMI interface with multi-agnet support, as shown
> > >> below.
> > >>
> > >> +-----------------------------------------+
> > >> | |
> > >> | EL3 TF-A SCMI |
> > >> +-------+--+-------+--+-------+--+-------++
> > >> |shmem0 | |shmem1 | |shmem2 | |shmemX |
> > >> +-----+-+ +---+---+ +--+----+ +---+---+
> > >> smc-id0 | | | |
> > >> agent0 | | | |
> > >> +-----v--------+---------+-----------+----+
> > >> | | | | |
> > >> | | | | |
> > >> +--------------+---------+-----------+----+
> > >> smc-id1 | smc-id2| smc-idX|
> > >> agent1 | agent2 | agentX |
> > >> | | |
> > >> +----v---+ +--v-----+ +--v-----+
> > >> | | | | | |
> > >> | Dom0 | | Dom1 | | DomX |
> > >> | | | | | |
> > >> | | | | | |
> > >> +--------+ +--------+ +--------+
> > >>
> > >> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
> > >> memory transport for every Agent in the system.
> > >>
> > >> The SCMI Agent transport channel defined by pair:
> > >> - smc-id: SMC/HVC id used for Doorbell
> > >> - shmem: shared memory for messages transfer, Xen page aligned,
> > >> p2m_mmio_direct_nc.
> > >>
> > >> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
> > >> multi-agent functionality under Xen:
> > >> - Xen manegement agent: trusted agents that accesses to the Base Protocol
> > >> commands to configure agent specific permissions
> > >> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
> > >> allowed direct HW access. At least one OSPM VM agent has to be provided
> > >> by FW if HW is handled only by Dom0 or Driver Domain.
> > >>
> > >> The EL3 SCMI FW expected to implement following Base protocol messages:
> > >> - BASE_DISCOVER_AGENT
> > >> - BASE_RESET_AGENT_CONFIGURATION (optional)
> > >> - BASE_SET_DEVICE_PERMISSIONS (optional)
> > >>
> > >> The SCI SCMI SMC multi-agent driver implements following functionality:
> > >> - It's initialized based on the Host DT SCMI node (only one SCMI interface
> > >> is supported) which describes Xen management agent SCMI interface.
> > >>
> > >> scmi_shm_0 : sram@47ff0000 {
> > >> compatible = "arm,scmi-shmem";
> > >> reg = <0x0 0x47ff0000 0x0 0x1000>;
> > >> };
> > >> firmware {
> > >> scmi: scmi {
> > >> compatible = "arm,scmi-smc";
> > >> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
> > > some extra spaces, it might be a copy/paste error
> > +
> > >> \#address-cells = < 1>;
> > >> \#size-cells = < 0>;
> > >> \#access-controller - cells = < 1>;
> > >> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
> > >>
> > >> protocol@X{
> > >> };
> > >> };
> > >> };
> > >>
> > >> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
> > >> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
> > >>
> > >> chosen {
> > >> xen,scmi-secondary-agents = <
> > >> 1 0x82000003 &scmi_shm_1
> > >> 2 0x82000004 &scmi_shm_2
> > >> 3 0x82000005 &scmi_shm_3
> > >> 4 0x82000006 &scmi_shm_4>;
> > >> }
> > >>
> > >> /{
> > >> scmi_shm_1: sram@47ff1000 {
> > >> compatible = "arm,scmi-shmem";
> > >> reg = <0x0 0x47ff1000 0x0 0x1000>;
> > >> };
> > >> scmi_shm_2: sram@47ff2000 {
> > >> compatible = "arm,scmi-shmem";
> > >> reg = <0x0 0x47ff2000 0x0 0x1000>;
> > >> };
> > >> scmi_shm_3: sram@47ff3000 {
> > >> compatible = "arm,scmi-shmem";
> > >> reg = <0x0 0x47ff3000 0x0 0x1000>;
> > >> };
> > >> }
> > >> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
> > >> this agent_id.
> > >>
> > >> Note that Xen is the only one entry in the system which need to know
> > >> about SCMI multi-agent support.
> > >>
> > >> - It implements the SCI subsystem interface required for configuring and
> > >> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
> > >> SCMI functionality for domain it has to be configured with unique supported
> > >> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
> > >> [smc-id, shmem] defined for this SCMI Agent_id.
> > >> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
> > >> -- zero-copy, the guest domain puts SCMI message in shmem;
> > >> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
> > >> -- the Xen driver catches exception, do checks and synchronously forwards
> > >> it to EL3 FW.
> > >> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
> > >> management agent channel on domain destroy event. This allows to reset
> > >> resources used by domain and so implement use-case like domain reboot.
> > >>
> > >> Dom0 Enable SCMI SMC:
> > >> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
> > >> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
> > >> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
> > >> node according to assigned agent_id.
> > >>
> > >> Guest domains enable SCMI SMC:
> > >> - xl.cfg: add configuration option as below
> > >>
> > >> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
> > >>
> > >> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
> > >> the domain, for example:
> > >>
> > >> iomem = [
> > >> "47ff2,1@22001",
> > >> ]
> > > Looking at the code and the configuration options, it looks like it is
> > > possible to map a scmi-shmem channel at a different address for the
> > > guest. It seems like it would work. Is that correct?
> > >
> > Yes it will. in our case address 22001000 should be the save as
> > sram@22001000 in the domain device-tree.
> > >> - DT: add SCMI nodes to the Driver domain partial device tree as in the
> > >> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
> > >>
> > >> passthrough {
> > >> scmi_shm_0: sram@22001000 {
> > >> compatible = "arm,scmi-shmem";
> > >> reg = <0x0 0x22001000 0x0 0x1000>;
> > >> };
> > >>
> > >> firmware {
> > >> compatible = "simple-bus";
> > >> scmi: scmi {
> > >> compatible = "arm,scmi-smc";
> > >> arm,smc-id = <0x82000004>;
> > >> shmem = <&scmi_shm_0>;
> > >> ...
> > >> }
> > >> }
> > >> }
> > >>
> > >> SCMI "4.2.1.1 Device specific access control"
> > >>
> > >> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
> > >> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
> > >> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
> > >> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
> > >> to the Xen SCMI.
> > >>
> > >> &i2c1 {
> > >> access-controllers = <&scmi 0>;
> > >> };
> > >>
> > >> The Dom0 and dom0less domains DT devices will be processed automatically through
> > >> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
> > >> shell be used:
> > >>
> > >> dtdev = [
> > >> "/soc/i2c@e6508000",
> > >> ]
> > >>
> > >> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
> > >>
> > >> [1]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
> > >> [2]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
> > >> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> > >> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> > > Thanks for the long explanation, great work! I am really looking forward
> > > to have this feature in the tree soon.
> > >
> > >
> > >> ---
> > >>
> > >> Changes in v4:
> > >> - toolstack comments from Anthony PERARD
> > >> - added dom0less support
> > >> - added doc for "xen,scmi-secondary-agents"
> > >>
> > >> docs/man/xl.cfg.5.pod.in | 13 +
> > >> docs/misc/arm/device-tree/booting.txt | 60 ++
> > >> docs/misc/xen-command-line.pandoc | 9 +
> > >> tools/libs/light/libxl_arm.c | 4 +
> > >> tools/libs/light/libxl_types.idl | 4 +-
> > >> tools/xl/xl_parse.c | 12 +
> > >> xen/arch/arm/dom0less-build.c | 11 +
> > >> xen/arch/arm/domain_build.c | 3 +-
> > >> xen/arch/arm/firmware/Kconfig | 11 +
> > >> xen/arch/arm/firmware/Makefile | 1 +
> > >> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
> > >> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
> > >> xen/arch/arm/firmware/scmi-shmem.h | 45 +
> > >> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
> > >> xen/include/public/arch-arm.h | 3 +
> > >> 15 files changed, 1371 insertions(+), 2 deletions(-)
> > >> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
> > >> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
> > >> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
> > >> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
> > >>
> > >> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> > >> index 1ccf50b8ea..302c46d8bc 100644
> > >> --- a/docs/man/xl.cfg.5.pod.in
> > >> +++ b/docs/man/xl.cfg.5.pod.in
> > >> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
> > >> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
> > >> option.
> > >>
> > >> +=item B<scmi_smc_multiagent>
> > >> +
> > >> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> > >> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
> > >> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
> > >> +specified for the guest.
> > >> +
> > >> =back
> > >>
> > >> +=item B<agent_id=NUMBER>
> > >> +
> > >> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
> > >> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
> > >> +existing on a single host must be unique and in the range [1..255].
> > >> +
> > >> =back
> > >>
> > >> =back
> > >> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> > >> index 8943c04173..c8923ab8b2 100644
> > >> --- a/docs/misc/arm/device-tree/booting.txt
> > >> +++ b/docs/misc/arm/device-tree/booting.txt
> > >> @@ -296,6 +296,20 @@ with the following properties:
> > >> Should be used together with dom0_scmi_smc_passthrough Xen command line
> > >> option.
> > >>
> > >> + - "scmi_smc_multiagent"
> > >> +
> > >> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> > >> + SMC calls forwarding from domain to the EL3 firmware (like ARM
> > >> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
> > >> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
> > >> + property.
> > >> +
> > >> +- "xen,sci_agent_id"
> > >> +
> > >> + Specifies a non-zero ARM SCI agent id for the guest. This option is
> > >> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
> > >> + the guest. The agent ids of guest must be unique and in the range [1..255].
> > >> +
> > >> Under the "xen,domain" compatible node, one or more sub-nodes are present
> > >> for the DomU kernel and ramdisk.
> > >>
> > >> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
> > >> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
> > >> guest physical address space. DomU1 is explicitly defined as the owner domain,
> > >> and DomU2 is the borrower domain.
> > >> +
> > >> +SCMI SMC multi-agent support
> > >> +============================
> > >> +
> > >> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
> > >> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
> > >> +according to the SCMI compliant EL3 Firmware specification with
> > >> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
> > >> +the top-level "chosen" node:
> > >> +
> > >> +- xen,scmi-secondary-agents
> > >> +
> > >> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
> > >> + available for Xen. Each Agent defined as triple consisting of:
> > >> + SCMI agent_id,
> > >> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
> > >> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
> > >> +
> > >> +As an example:
> > >> +
> > >> +chosen {
> > >> + xen,scmi-secondary-agents = <
> > >> + 1 0x82000003 &scmi_shm_1
> > >> + 2 0x82000004 &scmi_shm_2
> > >> + 3 0x82000005 &scmi_shm_3
> > >> + 4 0x82000006 &scmi_shm_4>;
> > >> +}
> > > NIT: it should be };
> > +
> > > Looking at scmi_probe, collect_agents, and the following SCMI
> > > SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
> > > information?
> > >
> > > It looks like we can discover the agend_ids for every channel, I guess
> > > what we need to know is the shmem location for every channel? But the
> > > full list of shmem channel is available below thanks to the scmi-shmem
> > > nodes.
> > >
> > > So, we have the list of scmi-shmem anyway, and we can probe the
> > > agent_id. The only parameter left is the smc_id/func_id.
> > >
> > > Or maybe smc_id/func_id can be calculated from agent_id?
> > >
> > > I am asking mostly because if a user is supposed to add this
> > > xen,scmi-secondary-agents property, where are they supposed to find the
> > > smc_id/func_id information?
> > >
> > > It is important that we write down in this document how the user is
> > > expected to find out what 1 is 0x82000003 which is scmi_shm_1.
> >
> > That's a very good question! The issue here is that there are no
> >
> > explicit requirements defining the relationship between agent_id and
> > func_id.
> >
> >
> > For example, in ARM-TF, different implementations can use different
> > func_ids.
> >
> > To provide better flexibility, we decided to separate agent_id from func_id.
> >
> >
> > Currently, the SCMI_BASE_DISCOVER_AGENT calls from the probe are intended to
> >
> > verify that all registered agents are present and that the configuration
> > is correct.
> >
> > However, I understand that this additional validation could be optional
> > to save traffic.
> >
> >
> > To address this, I’m considering adding a configuration option, such as
> >
> > CONFIG_SCMI_AGENT_VALIDATION, which can be disabled to reduce boot time
> > if this
> >
> > validation is not necessary for certain use cases.
> >
> >
> > Lastly, I’ll be updating the document to include clearer information
> > about the
> >
> > relationship between func_id and agent_id in the upcoming v5.
>
> The key point here is to make it easier for the user. If we can make
> agent_id or func_id optional it would make users lives easier.
>
> Alternative, or in addition to this, we should make the docs as clear as
> possible so that people can figure it out without having to ask
> questions on xen-devel.
>
>
> > >> +/{
> > >> + scmi_shm_1: sram@47ff1000 {
> > >> + compatible = "arm,scmi-shmem";
> > >> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> > >> + };
> > >> + scmi_shm_2: sram@47ff2000 {
> > >> + compatible = "arm,scmi-shmem";
> > >> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> > >> + };
> > >> + scmi_shm_3: sram@47ff3000 {
> > >> + compatible = "arm,scmi-shmem";
> > >> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> > >> + };
> > >> + scmi_shm_3: sram@47ff4000 {
> > >> + compatible = "arm,scmi-shmem";
> > >> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> > >> + };
> > > Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
> > > Or are under /firmware? Or are they under /chosen?
> > >
> > > I take they are under the top level node together with scmi_shm_0?
> > >
> > > Can you please also clarify in the document as well?
> > >
> > >
> > all these nodes are on the top level of the device-tree. But there is no
> > specific place for them.
> >
> > They could be subnodes to some memory-region for example. I will clarify
> > this.
> >
> > >> +}
> > >> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> > >> index 8e50f6b7c7..bc3c64d6ec 100644
> > >> --- a/docs/misc/xen-command-line.pandoc
> > >> +++ b/docs/misc/xen-command-line.pandoc
> > >> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
> > >> SCMI nodes removed from Dom0/hwdom device tree.
> > >> (for example, thin Dom0 with Driver domain use-case).
> > >>
> > >> +### dom0_scmi_agent_id (ARM)
> > >> +> `= <integer>`
> > >> +
> > >> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
> > >> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
> > >> +The SCMI will be disabled for Dom0 if this option is not specified
> > >> +(for example, thin Dom0 or dom0less use-cases).
> > >> +The agent ids of domains existing on a single host must be unique.
> > >> +
> > >> ### dtuart (ARM)
> > >> > `= path [:options]`
> > >>
> > >> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> > >> index 28ba9eb787..7712f53cd4 100644
> > >> --- a/tools/libs/light/libxl_arm.c
> > >> +++ b/tools/libs/light/libxl_arm.c
> > >> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> > >> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
> > >> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> > >> break;
> > >> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
> > >> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> > >> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
> > >> + break;
> > >> default:
> > >> LOG(ERROR, "Unknown ARM_SCI type %d",
> > >> d_config->b_info.arch_arm.arm_sci.type);
> > >> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> > >> index aa2190ab5b..11e31ce786 100644
> > >> --- a/tools/libs/light/libxl_types.idl
> > >> +++ b/tools/libs/light/libxl_types.idl
> > >> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
> > >>
> > >> libxl_arm_sci_type = Enumeration("arm_sci_type", [
> > >> (0, "none"),
> > >> - (1, "scmi_smc")
> > >> + (1, "scmi_smc"),
> > >> + (2, "scmi_smc_multiagent")
> > >> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
> > >>
> > >> libxl_arm_sci = Struct("arm_sci", [
> > >> ("type", libxl_arm_sci_type),
> > >> + ("agent_id", uint8)
> > >> ])
> > >>
> > >> libxl_rdm_reserve = Struct("rdm_reserve", [
> > >> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> > >> index bd22be9d33..81aa3797e3 100644
> > >> --- a/tools/xl/xl_parse.c
> > >> +++ b/tools/xl/xl_parse.c
> > >> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
> > >> }
> > >> }
> > >>
> > >> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
> > >> + unsigned long val = parse_ulong(oparg);
> > >> +
> > >> + if (!val || val > 255) {
> > >> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
> > >> + val);
> > >> + ret = ERROR_INVAL;
> > >> + goto parse_error;
> > >> + }
> > >> + arm_sci->agent_id = val;
> > >> + }
> > >> +
> > >> ptr = strtok(NULL, ",");
> > >> }
> > >>
> > >> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> > >> index 0a00f03a25..43d21eb889 100644
> > >> --- a/xen/arch/arm/dom0less-build.c
> > >> +++ b/xen/arch/arm/dom0less-build.c
> > >> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
> > >> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
> > >> else if ( !strcmp(sci_type, "scmi_smc") )
> > >> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
> > >> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
> > >> + {
> > >> + uint32_t agent_id = 0;
> > >> +
> > >> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
> > >> + !agent_id )
> > > shouldn't we check that agent_id <= 255 ?
> >
> > I see no limitation about max agent_id in DEN0056E document, it's uint32_t.
> >
> > >> + return -EINVAL;
> > >> +
> > >> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
> > >> + d_cfg->arch.arm_sci_agent_id = agent_id;
> > >> + }
> > >> else
> > >> {
> > >> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
> > >> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > >> index 36d28b52a4..0c9274a2b3 100644
> > >> --- a/xen/arch/arm/domain_build.c
> > >> +++ b/xen/arch/arm/domain_build.c
> > >> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
> > >> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
> > >> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
> > >> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
> > >> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
> > >> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
> > >> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
> > >> continue;
> > >>
> > >> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") ) diff --git a/xen/arch/arm/firmware/Kconfig
> > >> b/xen/arch/arm/firmware/Kconfig index 5c5f0880c4..6b051c8ada 100644
> > >> --- a/xen/arch/arm/firmware/Kconfig +++
> > >> b/xen/arch/arm/firmware/Kconfig @@ -29,6 +29,17 @@ config SCMI_SMC
> > >> driver domain. Use with EL3 firmware which supports only single SCMI
> > >> OSPM agent. +config SCMI_SMC_MA + bool "Enable ARM SCMI SMC multi-agent driver"
> > >> + select ARM_SCI
> > >> + help
> > >> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
> > >> + to EL3 firmware (TF-A) which supports multi-agent feature.
> > >> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
> > >> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
> > >> + allowed platform resources through dedicated SMC/HVC Shared memory based
> > >> + transport.
> > >> +
> > >> endchoice
> > >>
> > >> endmenu
> > >> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
> > >> index 71bdefc24a..37927e690e 100644
> > >> --- a/xen/arch/arm/firmware/Makefile
> > >> +++ b/xen/arch/arm/firmware/Makefile
> > >> @@ -1,2 +1,3 @@
> > >> obj-$(CONFIG_ARM_SCI) += sci.o
> > >> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
> > >> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
> > >> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
> > >> new file mode 100644
> > >> index 0000000000..3f4b9c5d6b
> > >> --- /dev/null
> > >> +++ b/xen/arch/arm/firmware/scmi-proto.h
> > >> @@ -0,0 +1,164 @@
> > >> +/* SPDX-License-Identifier: GPL-2.0-only */
> > >> +/*
> > >> + * Arm System Control and Management Interface definitions
> > >> + * Version 3.0 (DEN0056C)
> > >> + *
> > >> + * Copyright (c) 2024 EPAM Systems
> > >> + */
> > >> +
> > >> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> > >> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
> > > NIT: ARM_FIRMWARE_SCMI_PROTO_H
> > +
> > >> +#include <xen/stdint.h>
> > >> +
> > >> +#define SCMI_SHORT_NAME_MAX_SIZE 16
> > >> +
> > >> +/* SCMI status codes. See section 4.1.4 */
> > >> +#define SCMI_SUCCESS 0
> > >> +#define SCMI_NOT_SUPPORTED (-1)
> > >> +#define SCMI_INVALID_PARAMETERS (-2)
> > >> +#define SCMI_DENIED (-3)
> > >> +#define SCMI_NOT_FOUND (-4)
> > >> +#define SCMI_OUT_OF_RANGE (-5)
> > >> +#define SCMI_BUSY (-6)
> > >> +#define SCMI_COMMS_ERROR (-7)
> > >> +#define SCMI_GENERIC_ERROR (-8)
> > >> +#define SCMI_HARDWARE_ERROR (-9)
> > >> +#define SCMI_PROTOCOL_ERROR (-10)
> > >> +
> > >> +/* Protocol IDs */
> > >> +#define SCMI_BASE_PROTOCOL 0x10
> > >> +
> > >> +/* Base protocol message IDs */
> > >> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
> > >> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
> > >> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
> > >> +#define SCMI_BASE_DISCOVER_AGENT 0x7
> > >> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
> > >> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
> > >> +
> > >> +typedef struct scmi_msg_header {
> > >> + uint8_t id;
> > >> + uint8_t type;
> > >> + uint8_t protocol;
> > >> + uint32_t status;
> > >> +} scmi_msg_header_t;
> > >> +
> > >> +/* Table 2 Message header format */
> > >> +#define SCMI_HDR_ID GENMASK(7, 0)
> > >> +#define SCMI_HDR_TYPE GENMASK(9, 8)
> > >> +#define SCMI_HDR_PROTO GENMASK(17, 10)
> > >> +
> > >> +#define SCMI_FIELD_GET(_mask, _reg) \
> > >> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
> > >> +#define SCMI_FIELD_PREP(_mask, _val) \
> > >> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
> > >> +
> > >> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
> > >> +{
> > >> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
> > >> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
> > >> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
> > >> +}
> > >> +
> > >> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
> > >> +{
> > >> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
> > >> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
> > >> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
> > >> +}
> > >> +
> > >> +static inline int scmi_to_xen_errno(int scmi_status)
> > >> +{
> > >> + if ( scmi_status == SCMI_SUCCESS )
> > >> + return 0;
> > >> +
> > >> + switch ( scmi_status )
> > >> + {
> > >> + case SCMI_NOT_SUPPORTED:
> > >> + return -EOPNOTSUPP;
> > >> + case SCMI_INVALID_PARAMETERS:
> > >> + return -EINVAL;
> > >> + case SCMI_DENIED:
> > >> + return -EACCES;
> > >> + case SCMI_NOT_FOUND:
> > >> + return -ENOENT;
> > >> + case SCMI_OUT_OF_RANGE:
> > >> + return -ERANGE;
> > >> + case SCMI_BUSY:
> > >> + return -EBUSY;
> > >> + case SCMI_COMMS_ERROR:
> > >> + return -ENOTCONN;
> > >> + case SCMI_GENERIC_ERROR:
> > >> + return -EIO;
> > >> + case SCMI_HARDWARE_ERROR:
> > >> + return -ENXIO;
> > >> + case SCMI_PROTOCOL_ERROR:
> > >> + return -EBADMSG;
> > >> + default:
> > >> + return -EINVAL;
> > >> + }
> > >> +}
> > >> +
> > >> +/* PROTOCOL_VERSION */
> > >> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
> > >> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
> > >> +
> > >> +struct scmi_msg_prot_version_p2a {
> > >> + uint32_t version;
> > >> +} __packed;
> > >> +
> > >> +/* BASE PROTOCOL_ATTRIBUTES */
> > >> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
> > >> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
> > >> +
> > >> +struct scmi_msg_base_attributes_p2a {
> > >> + uint32_t attributes;
> > >> +} __packed;
> > >> +
> > >> +/*
> > >> + * BASE_DISCOVER_AGENT
> > >> + */
> > >> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
> > >> +
> > >> +struct scmi_msg_base_discover_agent_a2p {
> > >> + uint32_t agent_id;
> > >> +} __packed;
> > >> +
> > >> +struct scmi_msg_base_discover_agent_p2a {
> > >> + uint32_t agent_id;
> > >> + char name[SCMI_SHORT_NAME_MAX_SIZE];
> > >> +} __packed;
> > >> +
> > >> +/*
> > >> + * BASE_SET_DEVICE_PERMISSIONS
> > >> + */
> > >> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
> > >> +
> > >> +struct scmi_msg_base_set_device_permissions_a2p {
> > >> + uint32_t agent_id;
> > >> + uint32_t device_id;
> > >> + uint32_t flags;
> > >> +} __packed;
> > >> +
> > >> +/*
> > >> + * BASE_RESET_AGENT_CONFIGURATION
> > >> + */
> > >> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
> > >> +
> > >> +struct scmi_msg_base_reset_agent_cfg_a2p {
> > >> + uint32_t agent_id;
> > >> + uint32_t flags;
> > >> +} __packed;
> > >> +
> > >> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
> > >> +
> > >> +/*
> > >> + * Local variables:
> > >> + * mode: C
> > >> + * c-file-style: "BSD"
> > >> + * c-basic-offset: 4
> > >> + * tab-width: 4
> > >> + * indent-tabs-mode: nil
> > >> + * End:
> > >> + */
> > >> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
> > >> new file mode 100644
> > >> index 0000000000..dd613ee0b5
> > >> --- /dev/null
> > >> +++ b/xen/arch/arm/firmware/scmi-shmem.c
> > >> @@ -0,0 +1,173 @@
> > >> +/* SPDX-License-Identifier: GPL-2.0-only */
> > >> +/*
> > >> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
> > >> + *
> > >> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> > >> + * Copyright (c) 2025 EPAM Systems
> > >> + */
> > >> +/* SPDX-License-Identifier: GPL-2.0-only */
> > >> +
> > >> +#include <asm/io.h>
> > >> +#include <xen/err.h>
> > >> +
> > >> +#include "scmi-proto.h"
> > >> +#include "scmi-shmem.h"
> > > This code is written more generically than the description implies. If
> > > we only want to make SMC calls to TF-A on EL3 and exchange data with it
> > > over shared memory, then I think:
> > > - we don't need the __iomem tag, as there is no MMIO
> > > - we only need a DMB, not a DSB (readl and writel imply DSB, use only
> > > readl_relaxed and writel_relaxed)
> > >
> > > On the other hand, if we also want to handle the case where the SCMI
> > > server could be on a separate co-processor, then what this code is doing
> > > is not sufficient because we also need a dcache flush, in addition to
> > > the DSB.
> > >
> > > Bertrand, can you double-check?
> > >
> > >
> > >> +/*
> > >> + * Copy data from IO memory space to "real" memory space.
> > >> + */
> > >> +static void __memcpy_fromio(void *to, const volatile void __iomem *from,
> > >> + size_t count)
> > >> +{
> > >> + while ( count && !IS_ALIGNED((unsigned long)from, 4) )
> > >> + {
> > >> + *(u8 *)to = readb_relaxed(from);
> > >> + from++;
> > >> + to++;
> > >> + count--;
> > >> + }
> > >> +
> > >> + while ( count >= 4 )
> > >> + {
> > >> + *(u32 *)to = readl_relaxed(from);
> > >> + from += 4;
> > >> + to += 4;
> > >> + count -= 4;
> > >> + }
> > >> +
> > >> + while ( count )
> > >> + {
> > >> + *(u8 *)to = readb_relaxed(from);
> > >> + from++;
> > >> + to++;
> > >> + count--;
> > >> + }
> > >> +}
> > >> +
> > >> +/*
> > >> + * Copy data from "real" memory space to IO memory space.
> > >> + */
> > >> +static void __memcpy_toio(volatile void __iomem *to, const void *from,
> > >> + size_t count)
> > >> +{
> > >> + while ( count && !IS_ALIGNED((unsigned long)to, 4) )
> > >> + {
> > >> + writeb_relaxed(*(u8 *)from, to);
> > >> + from++;
> > >> + to++;
> > >> + count--;
> > >> + }
> > >> +
> > >> + while ( count >= 4 )
> > >> + {
> > >> + writel_relaxed(*(u32 *)from, to);
> > >> + from += 4;
> > >> + to += 4;
> > >> + count -= 4;
> > >> + }
> > >> +
> > >> + while ( count )
> > >> + {
> > >> + writeb_relaxed(*(u8 *)from, to);
> > >> + from++;
> > >> + to++;
> > >> + count--;
> > >> + }
> > >> +}
> > > I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
> > > we use a simple memcpy?
> >
> > This approach was used because we're trying to access shared memory
> > between two independent systems:
> >
> > Arm-TF and Xen in our case which places some chunks of data to the same
> > memory. And, according to the [0]
> >
> > ```
> >
> > Some devices (such as framebuffers) would like to use larger transfers than
> > 8 bytes at a time. For these devices, the memcpy_toio(),
> > memcpy_fromio() and memset_io() functions are
> > provided. Do not use memset or memcpy on IO addresses; they are not
> > guaranteed to copy data in order.
> >
> > ```
> >
> > Also, the same approach was used by Arm team when introducing scmi
> > driver to the Linux kernel [1]
> >
> >
> > [0]: https://www.kernel.org/doc/Documentation/driver-api/device-io.rst
> >
> > [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
>
>
> Keep in mind that [0] refers specifically to access to MMIO regions. I
> assume that the SCMI shared buffers are on normal memory? Regarding [1],
> it makes sense if Linux is trying to support shared memory over MMIO.
>
> Looking at one of your replies below, I am guessing the memory buffers
> are actually in normal memory but the issue is that TF-A is mapping them
> as uncacheable. Is that correct?
>
> In that case, I still don't understand why a simple memcpy would not be
> sufficient. Can you check?
>
> If yes, then for now I would just simplify it down to memcpy. When
> someone adds support for an SCMI server elsewhere we could look into
> adding a more sophisticated memcpy and we can look at the details at
> that point in time. Specifically, I am not convinced that memcpy_toio
> and memcpy_fromio would work if the SCMI server is on a separate
> non-coherent microcontroller.
See my other reply: https://marc.info/?l=xen-devel&m=175020352903542
I think we should implement this as:
memcpy(to, from, count); dsb(st or ld);
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-12 11:42 ` Oleksii Moisieiev
2025-06-12 13:10 ` Grygorii Strashko
@ 2025-06-18 0:04 ` Stefano Stabellini
2025-06-19 16:15 ` Oleksii Moisieiev
1 sibling, 1 reply; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-18 0:04 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: Stefano Stabellini, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Julien Grall, Michal Orzel, Roger Pau Monné,
Volodymyr Babchuk, Grygorii Strashko
[-- Attachment #1: Type: text/plain, Size: 10244 bytes --]
On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> Hi Stefano,
>
> I'm very sorry for a long silence. Please see my answers below:
>
> On 22/05/2025 03:25, Stefano Stabellini wrote:
> > On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> >> From: Grygorii Strashko<grygorii_strashko@epam.com>
> >>
> >> Add chained handling of assigned DT devices to support access-controller
> >> functionality through SCI framework, so DT device assign request can be
> >> passed to FW for processing and enabling VM access to requested device
> >> (for example, device power management through FW interface like SCMI).
> >>
> >> The SCI access-controller DT device processing is chained after IOMMU
> >> processing and expected to be executed for any DT device regardless of its
> >> protection by IOMMU (or if IOMMU is disabled).
> >>
> >> This allows to pass not only IOMMU protected DT device through
> >> xl.cfg:"dtdev" property for processing:
> >>
> >> dtdev = [
> >> "/soc/video@e6ef0000", <- IOMMU protected device
> >> "/soc/i2c@e6508000", <- not IOMMU protected device
> >> ]
> >>
> >> The change is done in two parts:
> >> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
> >> not fail if DT device is not protected by IOMMU
> >> 2) add chained call to sci_do_domctl() in do_domctl()
> >>
> >> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> >> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >> ---
> >>
> >>
> >>
> >> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
> >> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
> >> xen/common/domctl.c | 19 +++++++++++++
> >> xen/drivers/passthrough/device_tree.c | 6 ++++
> >> 4 files changed, 76 insertions(+)
> >>
> >> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
> >> index e1522e10e2..8efd541c4f 100644
> >> --- a/xen/arch/arm/firmware/sci.c
> >> +++ b/xen/arch/arm/firmware/sci.c
> >> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
> >> return 0;
> >> }
> >>
> >> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> >> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> >> +{
> >> + struct dt_device_node *dev;
> >> + int ret = 0;
> >> +
> >> + switch ( domctl->cmd )
> >> + {
> >> + case XEN_DOMCTL_assign_device:
> >> + ret = -EOPNOTSUPP;
> > Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>
> The -EOPNOTSUPP code is used because this is part of a chained call after
> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
> XEN_DOMCTL_assign_device
> call is expected to handle any DT device, regardless of whether the DT
> device is
> protected by an IOMMU or if the IOMMU is disabled.
> The following cases are considered:
>
> 1. IOMMU Protected Device (Success)
>
> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
> we continue
> processing the DT device by calling sci_do_domctl.
>
> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>
> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
> disabled,
> we still proceed to call sci_do_domctl.
OK this makes sense. I think it is OK to have a special error code to
say "the IOMMU is disabled" but I don't know if it is a good idea to try
to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
configuration with domctl disabled, for instance.
It might be wiser to use a different error code. Maybe ENOENT?
> 3. Error from iommu_do_domctl (Fail State)
>
> If iommu_do_domctl returns any error, the system enters a fail state, and
> sci_do_domctl is not called.
>
> 4. -EOPNOTSUPP from sci_do_domctl
>
> If sci_do_domctl returns -EOPNOTSUPP, this indicates one of the following:
> - The provided device is not a DT device.
> - There is no cur_mediator available (indicating that the SCI subsystem
> is enabled
> in the configuration, but no mediator was provided).
> - The current mediator does not support assign_dt_device (this is
> expected to be changed;
> see below for details).
> In this case, -EOPNOTSUPP is returned but will be ignored, and the
> original return value from iommu_do_domctl will be used as the final result.
Same comment as before. We need to be careful not confuse this case you
described with other cases where sci_do_domctl is simply not
implemented.
> 5. Return Code from sci_do_domctl
>
> If sci_do_domctl returns 0 (success) or an error code (failure),
> the return value from iommu_do_domctl is overridden, and the result from
> sci_do_domctl is returned.
> Note: -EOPNOTSUPP from iommu_do_domctl will also be overridden since
> step 2 was successfully completed (or failed).
> >> + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT )
> >> + break;
> > this one
> >
> >> + if ( !cur_mediator )
> >> + break;
> > this one
> >
> >> + if ( !cur_mediator->assign_dt_device )
> >> + break;
> > and also this one? It seems more like an -EINVAL as the caller used a
> > wrong parameter?
>
> I think you are right that this case should return -EINVAL because we
> should fail if mediator
>
> without implemented mandatory features was provided. Will be fixed.
>
> >> + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
> >> + domctl->u.assign_device.u.dt.size, &dev);
> >> + if ( ret )
> >> + return ret;
> >> +
> >> + ret = sci_assign_dt_device(d, dev);
> >> + if ( ret )
> >> + break;
> >> +
> >> + break;
> >> + default:
> >> + /* do not fail here as call is chained with iommu handling */
> > It looks like this should be an error
> >
> >
> >> + break;
> >> + }
> >> +
> >> + return ret;
> >> +}
> >> +
> >> static int __init sci_init(void)
> >> {
> >> struct dt_device_node *np;
> >> diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
> >> index 71fb54852e..b8d1bc8a62 100644
> >> --- a/xen/arch/arm/include/asm/firmware/sci.h
> >> +++ b/xen/arch/arm/include/asm/firmware/sci.h
> >> @@ -146,6 +146,14 @@ int sci_dt_finalize(struct domain *d, void *fdt);
> >> * control" functionality.
> >> */
> >> int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
> >> +
> >> +/*
> >> + * SCI domctl handler
> >> + *
> >> + * Only XEN_DOMCTL_assign_device is handled for now.
> >> + */
> >> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> >> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
> >> #else
> >>
> >> static inline bool sci_domain_is_enabled(struct domain *d)
> >> @@ -195,6 +203,12 @@ static inline int sci_assign_dt_device(struct domain *d,
> >> return 0;
> >> }
> >>
> >> +static inline int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> >> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> >> +{
> >> + return 0;
> >> +}
> >> +
> >> #endif /* CONFIG_ARM_SCI */
> >>
> >> #endif /* __ASM_ARM_SCI_H */
> >> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
> >> index 05abb581a0..a74ee92067 100644
> >> --- a/xen/common/domctl.c
> >> +++ b/xen/common/domctl.c
> >> @@ -27,6 +27,7 @@
> >> #include <xen/vm_event.h>
> >> #include <xen/monitor.h>
> >> #include <asm/current.h>
> >> +#include <asm/firmware/sci.h>
> >> #include <asm/irq.h>
> >> #include <asm/page.h>
> >> #include <asm/p2m.h>
> >> @@ -851,6 +852,24 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> >> case XEN_DOMCTL_deassign_device:
> >> case XEN_DOMCTL_get_device_group:
> >> ret = iommu_do_domctl(op, d, u_domctl);
> >> +
> >> + if ( !ret || ret == -EOPNOTSUPP )
> > It is better to invert the check:
> >
> > if ( ret < 0 && ret != -EOPNOTSUPP )
> > return ret;
> +
> >> + {
> >> + int ret1;
> >> + /*
> >> + * Add chained handling of assigned DT devices to support
> >> + * access-controller functionality through SCI framework, so
> >> + * DT device assign request can be passed to FW for processing and
> >> + * enabling VM access to requested device.
> >> + * The access-controller DT device processing is chained after IOMMU
> >> + * processing and expected to be executed for any DT device
> >> + * regardless if DT device is protected by IOMMU or not (or IOMMU
> >> + * is disabled).
> >> + */
> >> + ret1 = sci_do_domctl(op, d, u_domctl);
> >> + if ( ret1 != -EOPNOTSUPP )
> >> + ret = ret1;
> >> + }
> >> break;
> >>
> >> case XEN_DOMCTL_get_paging_mempool_size:
> >> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
> >> index 075fb25a37..2624767e51 100644
> >> --- a/xen/drivers/passthrough/device_tree.c
> >> +++ b/xen/drivers/passthrough/device_tree.c
> >> @@ -318,6 +318,12 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
> >> break;
> >> }
> >>
> >> + if ( !dt_device_is_protected(dev) )
> >> + {
> >> + ret = 0;
> >> + break;
> >> + }
> > I am concerned about this: previously we would call
> > iommu_assign_dt_device and the same check at the beginning of
> > iommu_assign_dt_device would return -EINVAL. Now it is a success.
> >
> > I am not sure this is appropriate. I wonder if instead we should:
> >
> > - remove this chunk from the patch
> > - change the return error for !dt_device_is_protected at the top of
> > iommu_assign_dt_device from -EINVAL to -EOPNOTSUPP
> > - this would fall into the same ret != -EOPNOTSUPP check after
> > iommu_do_domctl
>
> That's a good point. I think we should do the same for
>
> > if ( !is_iommu_enabled(d) )
>
> > return -EINVAL;
>
> because in this case we should process sci as well. I will do the change
>
> >> ret = iommu_assign_dt_device(d, dev);
> >>
> >> if ( ret )
> >> --
> >> 2.34.1
> >>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent
2025-06-12 12:03 ` Oleksii Moisieiev
@ 2025-06-18 0:35 ` Stefano Stabellini
2025-06-19 16:15 ` [RFC PATCH v4 8/8] docs: armproposa: l " Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-18 0:35 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: Stefano Stabellini, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Julien Grall, Michal Orzel, Roger Pau Monné,
Volodymyr Babchuk, Grygorii Strashko
[-- Attachment #1: Type: text/plain, Size: 16771 bytes --]
On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> On 23/05/2025 23:19, Stefano Stabellini wrote:
> > On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> >> From: Grygorii Strashko<grygorii_strashko@epam.com>
> >>
> >> Proposal description to add separate SCMI DT node for Xen management agent
> >> under "chosen" or xen-config node, like Hyperlaunch "xen,config".
> > I think it is OK to place a larger "xen,config" node under /chosen with
> > more information for Xen to setup SCMI more easily.
> >
> >
> >> This proposal introduces a new approach to the Xen multi-domain
> >> configuration, where all Xen-specific configuration has been moved
> >> under the "/chosen" node. This requires less Dom0 device tree
> >> manipulation and isolates Xen configuration from domain configuration.
> >>
> >> This approach provides the following device tree (DT) parameters:
> >>
> >> - "xen,scmi-secondary-agents": A Xen-specific parameter under the
> >> "/chosen" node, which describes the SCMI agent configuration for
> >> the domains.
> >> - the SCMI configuration for Xen (privileged agent) and the shared
> >> memory configuration for all agents are provided under the "/chosen"
> >> node and are used strictly by Xen for its initial configuration.
> >> - the scmi_shm and SCMI configuration for Dom0 are placed in the
> >> "/firmware/scmi" node so that they can be moved to Dom0 without
> >> any changes.
> > Isn't the SCMI configuration present in /firmware/scmi referring to the
> > privileged agent=0 meant to be used by Xen?
> >
> > I certainly see benefits in simplifying the configuration and especially
> > reducing the amount of changes a user might have to make on the
> > underlying device tree, but if the user needs to change /firmware/scmi
> > with the Dom0 information, it seems more dangerous and error prone than
> > the previous approach.
> >
> The idea is to move the privileged agent=0 configuration to the /chosen
> node and
>
> assign agent=1 to the Dom0 node under /firmware/scmi.
>
> Benefits of This Approach:
> - No Modification of the Xen DT Node Required
>
> This eliminates the need to modify the Xen Device Tree (DT) node
> before creating Dom0 in
>
> order to set the correct shared memory (shmem).
>
> -Consistent SCMI Configuration Format
>
> The Dom0 DT will have the same SCMI configuration format as other
> domains, simplifying the
>
> overall configuration process.
>
> - Unified SCMI Configuration Method
>
> There will no longer be a need to use a different approach for SCMI
> configuration in Dom0
>
> compared to other domains.
>
> - Separation Between Dom0 and Privileged Node
>
> This provides a clear separation between the Dom0 node and the
> privileged node.
>
> For example:
> If Dom0 only requires the clock protocol, but the Xen SCMI
> configuration requires additional protocols,
>
> this approach allows Dom0 to receive only the necessary
> protocol configuration.
I don't think this is a good idea because we end up confusing the data
for Xen and the data for the DomUs/Dom0 in the host device tree.
I think we should follow these very simple guidelines:
- The host DTB (the DTB given to Xen at boot) should be the same for Xen
and for Linux baremetal (no KVM), with the exception of the data under
the /chosen node
- We can place Xen specific configurations under the /chosen node in the
host DTB, both Xen hypervisor configuration and also Dom0/DomU
configurations
This way, the host information remains generic and the configuration for
Xen the domUs/Dom0 is kept clearly separate from the rest. I don't
think we can break these two assumptions but we have more freedom with
the rest.
If we start with these two simple assumptions, here are the
consequences:
- data under /firmware/scmi should be the same for Xen and baremetal
Linux, ideally it would describe Xen's agent0 channel in both cases
- We can add as many nodes as we like under /chosen, including a
xen,config node and also additional nodes for the domains config
- We can define the new nodes under /chosen to be as simple as possible
for the user to configure them, while also trying to minimize
complexity in Xen in terms of DT manipulations
If the Xen SCMI configuration data cannot be the same as the Linux
baremetal SCMI configuration (i.e. /firmware/scmi has to be different in
the two cases) I would still suggest to avoid modifying /firmware/scmi
for Xen and instead provide the Xen configuration under /chosen. It is
important to keep everything in the host DTB (except /chosen) the same
between Linux baremetal and Xen.
However, we can add a new node similar to /firmware/scmi under /chosen
specifically for Xen, such as /chosen/xen-config/scmi
The Dom0 configuration cannot be expected to be under /firmware/scmi.
However, it could also be defined under /chosen.
Keep in mind that the more we add to /chosen the more difficult it will
be for the user to configure the system. I think we should plan ahead to
have ImageBuilder be able to generate the DT nodes under /chosen for Xen
starting from the simplest possible configuration format provided by the
user. The more complex and rich are the device tree nodes under /chosen,
the more important is the documentation and ImageBuilder support for it.
> >> This configuration allows the use of Xen-specific nodes to provide
> >> information strictly needed by Xen while using the default SCMI
> >> configuration for Dom0 and other domains. As a result, no additional
> >> bindings need to be introduced to the device tree.
> > This is not actually implemented by this patch series, right?
> It is not. Just posted this document as a proposal.
> >> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> >> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >> ---
> >>
> >>
> >>
> >> .../arm/firmware/arm-scmi-proposal.rst | 224 ++++++++++++++++++
> >> 1 file changed, 224 insertions(+)
> >> create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> >>
> >> diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> >> new file mode 100644
> >> index 0000000000..fcc2ed2b65
> >> --- /dev/null
> >> +++ b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> >> @@ -0,0 +1,224 @@
> >> +
> >> +Proposal for SCMI multi-agent driver bindings
> >> +=============================================
> >> +
> >> +Now the Xen configuration for SCMI multi-agent support is done in a bit complicated way, especially
> >> +from SCMI multi-agent driver initialization and Dom0 DT manipulation point of view.
> >> +Also it does not take into account future requirements to support SCP SCMI FW.
> >> +
> >> +To enable SCMI multi-agent user need:
> >> +
> >> +* take host DT with basic SCMI enabled
> >> +* add SCMI shared-memory nodes for all agents
> >> +* update SCMI node to point on SCMI Xen management channel (``[smc-id, shmem]``)
> >> +* add "xen,scmi-secondary-agents" property to the "\chosen" node
> >> +
> >> +.. code::
> >> +
> >> + chosen {
> >> + xen,scmi-secondary-agents = <
> >> + 1 0x82000003 &scmi_shm_1
> >> + 2 0x82000004 &scmi_shm_2
> >> + 3 0x82000005 &scmi_shm_3
> >> + 4 0x82000006 &scmi_shm_4>;
> >> + }
> >> +
> >> + /{
> >> + // SCMI shared-memory nodes for all agents
> >> + scmi_shm_0 : sram@47ff0000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_1: sram@47ff1000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_2: sram@47ff2000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_3: sram@47ff3000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_4: sram@47ff4000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> >> + };
> >> +
> >> + firmware {
> >> + scmi: scmi {
> >> + compatible = "arm,scmi-smc";
> >> + arm, smc - id = <0x82000002>; <--- Xen management agent channel "smc-id"
> >> + #address-cells = < 1>;
> >> + #size-cells = < 0>;
> >> + #access-controller-cells = < 1>;
> >> + shmem = <&scmi_shm_0>; <--- Xen management agent channel "shmem"
> >> +
> >> + protocol@X{
> >> + };
> >> + };
> >> + };
> >> + }
> >> +
> >> +Important thing to note is that all information about multi-channel support is strictly Xen specific.
> >> +
> >> +During initialization the SCMI multi-agent driver uses Host DT SCMI node and
> >> +"xen,scmi-secondary-agents" property to init itself and then, during Dom0 creation, manipulates
> >> +Dom0 DT to remove Xen specific SCMI info and update dom0 SCMI nodes with Dom0 SCMI agent specific
> >> +information.
> >> +
> >> +There are two negative points:
> >> +
> >> +1) Double DT modification - one is user to set up SCMI Xen support in Host DT, second -
> >> + Dom0 DT manipulation.
> >> +2) In case of future support of mailbox shared-memory transport there could be up to 4 mailboxes and
> >> + up to 2 shared-memories per SCMI agent channel.
> >> +
> >> +Hence SCMI multi-agent support is Xen specific knowledge there is a proposal to add it as Xen
> >> +specific DT definitions and so minimize Host and Dom0 DT manipulations.
> >> +Those definitions can be added in "/chosen" or, ideally, in "xen,config" node (like in Hyperlaunch design).
> >> +
> >> +The SCMI binding stays generic, just two SCMI nodes defined - one for Xen management channel and
> >> +one for Host Dom0 OSPM.
> >> +
> >> +Example of using "chosen" for configuration:
> >> +
> >> +.. code::
> >> +
> >> + /{
> >> +
> >> + chosen {
> >> + ...
> >> +
> >> + // Xen SCMI management channel
> >> + scmi_shm_0 : sram@47ff0000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> >> + };
> >> + scmi_xen: scmi {
> >> + compatible = "arm,scmi-smc";
> >> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
> >> + #address-cells = < 1>;
> >> + #size-cells = < 0>;
> >> + #access-controller-cells = < 1>;
> >> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
> >> + };
> >> +
> >> + // SCMI multi-agent configuration
> >> + scmi_shm_2: sram@47ff2000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_3: sram@47ff3000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_4: sram@47ff4000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> >> + };
> >> + xen,scmi-secondary-agents = <
> >> + 1 0x82000003 &scmi_shm
> >> + 2 0x82000004 &scmi_shm_2
> >> + 3 0x82000005 &scmi_shm_3
> >> + 4 0x82000006 &scmi_shm_4>;
> >> + };
> >> +
> >> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
> >> + scmi_shm: sram@47ff1000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> >> + };
> >> +
> >> + firmware {
> >> + scmi: scmi {
> >> + compatible = "arm,scmi-smc";
> >> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
> >> + #address-cells = < 1>;
> >> + #size-cells = < 0>;
> >> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
> > By OSPM you mean Dom0 and not Xen? So this is a change compared to a
> > device tree for baremetal Linux without Xen?
> >
> > Let me ask the same question differently. In the case of barematal Linux
> > without Xen (no KVM), what would Linux see under /firmware/scmi as
> > smc-id and shmem? The same as the one that Xen would use for itself? Or
> > the same as the ones that Dom0 would use when Xen is present?
>
> If this DT is used with the baremetal Linux - then the Linux Kernel will
>
> see Dom0 "smc-id" and "shmen" under /firmware/scmi.
>
> >> + protocol@X{
> >> + };
> >> + };
> >> + };
> >> + }
> >> +
> >> +
> >> +In the above case:
> >> +
> >> +1) Xen SCMI multi-agent can be probed with DT configuration from "chosen" (or special "xen,config")
> >> + node and all Xen related nodes can be easily dropped from Dom0 DT.
> >> +2) Host SCMI OSPM channel DT nodes can be copied to Dom0 DT without changes if SCMI enabled for it.
> >> +3) Future support for mailbox shared-memory transport (SCP SCMI FW) can be simplified as no more
> >> + manipulation required with Dom0 SCMI "arm,smc-id" and "shmem" DT properties.
> > Yes, I can see the benefit if we can arrange it so that the underlying
> > host device tree is the same that Linux would use baremetal. And all the
> > extra configuration is placed under /chosen in "xen,config" node or
> > similar. I would probably call it "xen,scmi".
>
> Personally, I would keep "xen,config" as it leaves room to add additional
>
> configuration nodes in the future.
>
> >> +Example of using "xen,config" for configuration:
> >> +
> >> +.. code::
> >> +
> >> + hypervisor {
> >> + compatible = “hypervisor,xen”
> >> +
> >> + // Configuration container
> >> + config {
> >> + compatible = "xen,config";
> >> + ...
> >> +
> >> + // Xen SCMI management channel
> >> + scmi_shm_0 : sram@47ff0000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> >> + };
> >> + scmi_xen: scmi {
> >> + compatible = "arm,scmi-smc";
> >> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
> >> + #address-cells = < 1>;
> >> + #size-cells = < 0>;
> >> + #access-controller-cells = < 1>;
> >> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
> >> + };
> >> +
> >> + // SCMI multi-agent configuration
> >> + scmi_shm_2: sram@47ff2000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_3: sram@47ff3000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> >> + };
> >> + scmi_shm_4: sram@47ff4000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> >> + };
> >> + xen,scmi-secondary-agents = <
> >> + 1 0x82000003 &scmi_shm
> >> + 2 0x82000004 &scmi_shm_2
> >> + 3 0x82000005 &scmi_shm_3
> >> + 4 0x82000006 &scmi_shm_4>;
> >> + };
> >> + };
> >> +
> >> + /{
> >> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
> >> + scmi_shm: sram@47ff1000 {
> >> + compatible = "arm,scmi-shmem";
> >> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> >> + };
> >> +
> >> + firmware {
> >> + scmi: scmi {
> >> + compatible = "arm,scmi-smc";
> >> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
> >> + #address-cells = < 1>;
> >> + #size-cells = < 0>;
> >> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
> >> +
> >> + protocol@X{
> >> + };
> >> + };
> >> + };
> >> + }
> >> --
> >> 2.34.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-17 23:38 ` Stefano Stabellini
@ 2025-06-18 7:22 ` Julien Grall
2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-19 16:15 ` Oleksii Moisieiev
1 sibling, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-18 7:22 UTC (permalink / raw)
To: Stefano Stabellini, Grygorii Strashko
Cc: Bertrand Marquis, Oleksii Moisieiev,
xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Jan Beulich, Juergen Gross, Michal Orzel, Roger Pau Monné,
Volodymyr Babchuk
Hi,
On 18/06/2025 00:38, Stefano Stabellini wrote:
> On Thu, 12 Jun 2025, Grygorii Strashko wrote:
>> On 02.06.25 10:17, Bertrand Marquis wrote:
>>>> On the other hand, if we also want to handle the case where the SCMI
>>>> server could be on a separate co-processor, then what this code is doing
>>>> is not sufficient because we also need a dcache flush, in addition to
>>>> the DSB.
>>>>
>>>> Bertrand, can you double-check?
>>>
>>> If we want to handle a case where the memory is accessible to a coprocessor
>>> but there is no cache coherency, we need to flush the dcache definitely.
>>>
>>> Seeing the amount of data here, I do agree with Stefano that it would be a
>>> good
>>> idea to make the provision to flush the data cache in all cases. Even if the
>>> data
>>> is accessed by a secure partition or the firmware coherently, flushing in
>>> all cases
>>> would have very limited performance impact here.
>>>
>>> There is the other solution to have some kind of parameter to say if the
>>> accessor
>>> has coherent cache access but I do not think the performance impact here
>>> would
>>> justify such a complexity.
>>>
>> The SCMI shmem expected to be mapped as MT_NON_CACHEABLE in all cases.
I can't find MT_NON_CACHEABLE anywhere in Xen or Linux. My
interpretation is that the memory attribute would be normal memory non
cacheable. However, this doesn't add up with ...
>> The Linux does devm_ioremap() -> ioremap() ->
>> (ARM64) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
... this line. This is device nGnRE which is a lot more restrictive (for
instance it doesn't allow unaligned access).
>>
>> There is also note in docs:
>> "+- shmem: shared memory for messages transfer, **Xen page aligned** with
>> mapping``p2m_mmio_direct_nc``."
>>
>> In the case of SCP - the SCMI shmem can be actually be in SRAM.
>>
>> So, are you sure cache manipulations are required here?
>
> No, if the memory is mapped as uncacheable everywhere then the cache
> manipulations are not needed. However, we probably still need a dsb.
>
> I understand now why they decided to use __memcpy_fromio in Linux: it is
> not MMIO but they needed a memcpy followed by DSB, so they decided to
> reuse the existing MMIO functions although the buffer is not MMIO.
From my understanding, memcpy_fromio() is not just a mempcy() + dsb. It
also guarantees the access will be aligned (this is not guarantee by our
memcpy()).
Now the question is why does Linux map the region Device nGnRE but we
are mapping non-cacheable?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-05-19 15:50 ` [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver Oleksii Moisieiev
2025-05-23 20:06 ` Stefano Stabellini
@ 2025-06-18 7:28 ` Julien Grall
2025-06-19 16:15 ` Oleksii Moisieiev
1 sibling, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-18 7:28 UTC (permalink / raw)
To: Oleksii Moisieiev, xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Michal Orzel, Roger Pau Monné,
Stefano Stabellini, Volodymyr Babchuk, Grygorii Strashko
Hi Oleksii,
On 19/05/2025 16:50, Oleksii Moisieiev wrote:
> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
> (TF-A) which provides SCMI interface with multi-agnet support, as shown
s/multi-agnet/multi-agent/
> below.
>
> +-----------------------------------------+
> | |
> | EL3 TF-A SCMI |
> +-------+--+-------+--+-------+--+-------++
> |shmem0 | |shmem1 | |shmem2 | |shmemX |
> +-----+-+ +---+---+ +--+----+ +---+---+
> smc-id0 | | | |
> agent0 | | | |
> +-----v--------+---------+-----------+----+
> | | | | |
> | | | | |
> +--------------+---------+-----------+----+
> smc-id1 | smc-id2| smc-idX|
> agent1 | agent2 | agentX |
> | | |
> +----v---+ +--v-----+ +--v-----+
> | | | | | |
> | Dom0 | | Dom1 | | DomX |
> | | | | | |
> | | | | | |
> +--------+ +--------+ +--------+
>
> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
> memory transport for every Agent in the system.
>
> The SCMI Agent transport channel defined by pair:
> - smc-id: SMC/HVC id used for Doorbell
> - shmem: shared memory for messages transfer, Xen page aligned,
> p2m_mmio_direct_nc.
It is not clear why we nention Xen page aligned and p2m_mmio_direct_nc.
Is this multi-agent protocol tied to Xen?
That said... p2m_mmio_direct_nc is a type used in the stage 2
page-tables to indicate how we restrict access from the domain.
The resulting memory attribute will be a combination of stage-1 +
stage-2. In the future, we may decide to use FWB which will allow Xen to
force a specific memory attribute.
This is also purely internal decision. In the documentation, you should
spell out the memory attribute that should be used. From the discussion
on this patch, it is still unclear whether the region should be mapped
as Device nGnRE or normal memory non-cacheabl.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-17 23:22 ` Stefano Stabellini
2025-06-17 23:56 ` Stefano Stabellini
@ 2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 22:15 ` Stefano Stabellini
1 sibling, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-19 16:15 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
On 18/06/2025 02:22, Stefano Stabellini wrote:
> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>> Hi Stefano,
>>
>> I'm very sorry for a long silence. Please see my answers below:
>>
>>
>> On 23/05/2025 23:06, Stefano Stabellini wrote:
>>> One question for Bertrand below
>>>
>>>
>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
>>>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
>>>> below.
>>>>
>>>> +-----------------------------------------+
>>>> | |
>>>> | EL3 TF-A SCMI |
>>>> +-------+--+-------+--+-------+--+-------++
>>>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
>>>> +-----+-+ +---+---+ +--+----+ +---+---+
>>>> smc-id0 | | | |
>>>> agent0 | | | |
>>>> +-----v--------+---------+-----------+----+
>>>> | | | | |
>>>> | | | | |
>>>> +--------------+---------+-----------+----+
>>>> smc-id1 | smc-id2| smc-idX|
>>>> agent1 | agent2 | agentX |
>>>> | | |
>>>> +----v---+ +--v-----+ +--v-----+
>>>> | | | | | |
>>>> | Dom0 | | Dom1 | | DomX |
>>>> | | | | | |
>>>> | | | | | |
>>>> +--------+ +--------+ +--------+
>>>>
>>>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
>>>> memory transport for every Agent in the system.
>>>>
>>>> The SCMI Agent transport channel defined by pair:
>>>> - smc-id: SMC/HVC id used for Doorbell
>>>> - shmem: shared memory for messages transfer, Xen page aligned,
>>>> p2m_mmio_direct_nc.
>>>>
>>>> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
>>>> multi-agent functionality under Xen:
>>>> - Xen manegement agent: trusted agents that accesses to the Base Protocol
>>>> commands to configure agent specific permissions
>>>> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
>>>> allowed direct HW access. At least one OSPM VM agent has to be provided
>>>> by FW if HW is handled only by Dom0 or Driver Domain.
>>>>
>>>> The EL3 SCMI FW expected to implement following Base protocol messages:
>>>> - BASE_DISCOVER_AGENT
>>>> - BASE_RESET_AGENT_CONFIGURATION (optional)
>>>> - BASE_SET_DEVICE_PERMISSIONS (optional)
>>>>
>>>> The SCI SCMI SMC multi-agent driver implements following functionality:
>>>> - It's initialized based on the Host DT SCMI node (only one SCMI interface
>>>> is supported) which describes Xen management agent SCMI interface.
>>>>
>>>> scmi_shm_0 : sram@47ff0000 {
>>>> compatible = "arm,scmi-shmem";
>>>> reg = <0x0 0x47ff0000 0x0 0x1000>;
>>>> };
>>>> firmware {
>>>> scmi: scmi {
>>>> compatible = "arm,scmi-smc";
>>>> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
>>> some extra spaces, it might be a copy/paste error
>> +
>>>> \#address-cells = < 1>;
>>>> \#size-cells = < 0>;
>>>> \#access-controller - cells = < 1>;
>>>> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
>>>>
>>>> protocol@X{
>>>> };
>>>> };
>>>> };
>>>>
>>>> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
>>>> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
>>>>
>>>> chosen {
>>>> xen,scmi-secondary-agents = <
>>>> 1 0x82000003 &scmi_shm_1
>>>> 2 0x82000004 &scmi_shm_2
>>>> 3 0x82000005 &scmi_shm_3
>>>> 4 0x82000006 &scmi_shm_4>;
>>>> }
>>>>
>>>> /{
>>>> scmi_shm_1: sram@47ff1000 {
>>>> compatible = "arm,scmi-shmem";
>>>> reg = <0x0 0x47ff1000 0x0 0x1000>;
>>>> };
>>>> scmi_shm_2: sram@47ff2000 {
>>>> compatible = "arm,scmi-shmem";
>>>> reg = <0x0 0x47ff2000 0x0 0x1000>;
>>>> };
>>>> scmi_shm_3: sram@47ff3000 {
>>>> compatible = "arm,scmi-shmem";
>>>> reg = <0x0 0x47ff3000 0x0 0x1000>;
>>>> };
>>>> }
>>>> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
>>>> this agent_id.
>>>>
>>>> Note that Xen is the only one entry in the system which need to know
>>>> about SCMI multi-agent support.
>>>>
>>>> - It implements the SCI subsystem interface required for configuring and
>>>> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
>>>> SCMI functionality for domain it has to be configured with unique supported
>>>> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
>>>> [smc-id, shmem] defined for this SCMI Agent_id.
>>>> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
>>>> -- zero-copy, the guest domain puts SCMI message in shmem;
>>>> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
>>>> -- the Xen driver catches exception, do checks and synchronously forwards
>>>> it to EL3 FW.
>>>> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
>>>> management agent channel on domain destroy event. This allows to reset
>>>> resources used by domain and so implement use-case like domain reboot.
>>>>
>>>> Dom0 Enable SCMI SMC:
>>>> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
>>>> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
>>>> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
>>>> node according to assigned agent_id.
>>>>
>>>> Guest domains enable SCMI SMC:
>>>> - xl.cfg: add configuration option as below
>>>>
>>>> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
>>>>
>>>> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
>>>> the domain, for example:
>>>>
>>>> iomem = [
>>>> "47ff2,1@22001",
>>>> ]
>>> Looking at the code and the configuration options, it looks like it is
>>> possible to map a scmi-shmem channel at a different address for the
>>> guest. It seems like it would work. Is that correct?
>>>
>> Yes it will. in our case address 22001000 should be the save as
>> sram@22001000 in the domain device-tree.
>>>> - DT: add SCMI nodes to the Driver domain partial device tree as in the
>>>> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
>>>>
>>>> passthrough {
>>>> scmi_shm_0: sram@22001000 {
>>>> compatible = "arm,scmi-shmem";
>>>> reg = <0x0 0x22001000 0x0 0x1000>;
>>>> };
>>>>
>>>> firmware {
>>>> compatible = "simple-bus";
>>>> scmi: scmi {
>>>> compatible = "arm,scmi-smc";
>>>> arm,smc-id = <0x82000004>;
>>>> shmem = <&scmi_shm_0>;
>>>> ...
>>>> }
>>>> }
>>>> }
>>>>
>>>> SCMI "4.2.1.1 Device specific access control"
>>>>
>>>> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
>>>> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
>>>> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
>>>> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
>>>> to the Xen SCMI.
>>>>
>>>> &i2c1 {
>>>> access-controllers = <&scmi 0>;
>>>> };
>>>>
>>>> The Dom0 and dom0less domains DT devices will be processed automatically through
>>>> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
>>>> shell be used:
>>>>
>>>> dtdev = [
>>>> "/soc/i2c@e6508000",
>>>> ]
>>>>
>>>> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
>>>>
>>>> [1]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
>>>> [2]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>> Thanks for the long explanation, great work! I am really looking forward
>>> to have this feature in the tree soon.
>>>
>>>
>>>> ---
>>>>
>>>> Changes in v4:
>>>> - toolstack comments from Anthony PERARD
>>>> - added dom0less support
>>>> - added doc for "xen,scmi-secondary-agents"
>>>>
>>>> docs/man/xl.cfg.5.pod.in | 13 +
>>>> docs/misc/arm/device-tree/booting.txt | 60 ++
>>>> docs/misc/xen-command-line.pandoc | 9 +
>>>> tools/libs/light/libxl_arm.c | 4 +
>>>> tools/libs/light/libxl_types.idl | 4 +-
>>>> tools/xl/xl_parse.c | 12 +
>>>> xen/arch/arm/dom0less-build.c | 11 +
>>>> xen/arch/arm/domain_build.c | 3 +-
>>>> xen/arch/arm/firmware/Kconfig | 11 +
>>>> xen/arch/arm/firmware/Makefile | 1 +
>>>> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
>>>> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
>>>> xen/arch/arm/firmware/scmi-shmem.h | 45 +
>>>> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
>>>> xen/include/public/arch-arm.h | 3 +
>>>> 15 files changed, 1371 insertions(+), 2 deletions(-)
>>>> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
>>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
>>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
>>>> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
>>>>
>>>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>>>> index 1ccf50b8ea..302c46d8bc 100644
>>>> --- a/docs/man/xl.cfg.5.pod.in
>>>> +++ b/docs/man/xl.cfg.5.pod.in
>>>> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
>>>> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
>>>> option.
>>>>
>>>> +=item B<scmi_smc_multiagent>
>>>> +
>>>> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>>>> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
>>>> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
>>>> +specified for the guest.
>>>> +
>>>> =back
>>>>
>>>> +=item B<agent_id=NUMBER>
>>>> +
>>>> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
>>>> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
>>>> +existing on a single host must be unique and in the range [1..255].
>>>> +
>>>> =back
>>>>
>>>> =back
>>>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
>>>> index 8943c04173..c8923ab8b2 100644
>>>> --- a/docs/misc/arm/device-tree/booting.txt
>>>> +++ b/docs/misc/arm/device-tree/booting.txt
>>>> @@ -296,6 +296,20 @@ with the following properties:
>>>> Should be used together with dom0_scmi_smc_passthrough Xen command line
>>>> option.
>>>>
>>>> + - "scmi_smc_multiagent"
>>>> +
>>>> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
>>>> + SMC calls forwarding from domain to the EL3 firmware (like ARM
>>>> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
>>>> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
>>>> + property.
>>>> +
>>>> +- "xen,sci_agent_id"
>>>> +
>>>> + Specifies a non-zero ARM SCI agent id for the guest. This option is
>>>> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
>>>> + the guest. The agent ids of guest must be unique and in the range [1..255].
>>>> +
>>>> Under the "xen,domain" compatible node, one or more sub-nodes are present
>>>> for the DomU kernel and ramdisk.
>>>>
>>>> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
>>>> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
>>>> guest physical address space. DomU1 is explicitly defined as the owner domain,
>>>> and DomU2 is the borrower domain.
>>>> +
>>>> +SCMI SMC multi-agent support
>>>> +============================
>>>> +
>>>> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
>>>> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
>>>> +according to the SCMI compliant EL3 Firmware specification with
>>>> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
>>>> +the top-level "chosen" node:
>>>> +
>>>> +- xen,scmi-secondary-agents
>>>> +
>>>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
>>>> + available for Xen. Each Agent defined as triple consisting of:
>>>> + SCMI agent_id,
>>>> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
>>>> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
>>>> +
>>>> +As an example:
>>>> +
>>>> +chosen {
>>>> + xen,scmi-secondary-agents = <
>>>> + 1 0x82000003 &scmi_shm_1
>>>> + 2 0x82000004 &scmi_shm_2
>>>> + 3 0x82000005 &scmi_shm_3
>>>> + 4 0x82000006 &scmi_shm_4>;
>>>> +}
>>> NIT: it should be };
>> +
>>> Looking at scmi_probe, collect_agents, and the following SCMI
>>> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
>>> information?
>>>
>>> It looks like we can discover the agend_ids for every channel, I guess
>>> what we need to know is the shmem location for every channel? But the
>>> full list of shmem channel is available below thanks to the scmi-shmem
>>> nodes.
>>>
>>> So, we have the list of scmi-shmem anyway, and we can probe the
>>> agent_id. The only parameter left is the smc_id/func_id.
>>>
>>> Or maybe smc_id/func_id can be calculated from agent_id?
>>>
>>> I am asking mostly because if a user is supposed to add this
>>> xen,scmi-secondary-agents property, where are they supposed to find the
>>> smc_id/func_id information?
>>>
>>> It is important that we write down in this document how the user is
>>> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
>> That's a very good question! The issue here is that there are no
>>
>> explicit requirements defining the relationship between agent_id and
>> func_id.
>>
>>
>> For example, in ARM-TF, different implementations can use different
>> func_ids.
>>
>> To provide better flexibility, we decided to separate agent_id from func_id.
>>
>>
>> Currently, the SCMI_BASE_DISCOVER_AGENT calls from the probe are intended to
>>
>> verify that all registered agents are present and that the configuration
>> is correct.
>>
>> However, I understand that this additional validation could be optional
>> to save traffic.
>>
>>
>> To address this, I’m considering adding a configuration option, such as
>>
>> CONFIG_SCMI_AGENT_VALIDATION, which can be disabled to reduce boot time
>> if this
>>
>> validation is not necessary for certain use cases.
>>
>>
>> Lastly, I’ll be updating the document to include clearer information
>> about the
>>
>> relationship between func_id and agent_id in the upcoming v5.
>
> The key point here is to make it easier for the user. If we can make
> agent_id or func_id optional it would make users lives easier.
>
> Alternative, or in addition to this, we should make the docs as clear as
> possible so that people can figure it out without having to ask
> questions on xen-devel.
>
>
[OM ANSWER]
SCMI_BASE_DISCOVER_AGENT message requires agent_id. I think we can calculate
agent_id from func_id, the documentation will look like this:
```
For enabling the ARM SCMI SMC multi-agent support (enabled by
CONFIG_SCMI_SMC_MA)
the Xen specific SCMI Agent's configuration shell be provided in the Host DT
according to the SCMI compliant EL3 Firmware specification with
ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
the top-level "chosen" node:
- xen,scmi-secondary-agents
Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
available for Xen. Each Agent defined as triple consisting of:
SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
phandle to SCMI SHM assigned for the agent transport
("arm,scmi-shmem"),
SCMI agent_id (optional)
As an example:
chosen {
xen,scmi-secondary-agents = <
0x82000003 &scmi_shm_1 1
0x82000004 &scmi_shm_2 2
0x82000005 &scmi_shm_3 3
0x82000006 &scmi_shm_4 4>;
}
or
chosen {
xen,scmi-secondary-agents = <
0x82000003 &scmi_shm_1
0x82000004 &scmi_shm_2
0x82000005 &scmi_shm_3
0x82000006 &scmi_shm_4>;
}
If the optional parameter agent_id is not provided, then Xen will
request agent_id for each registered
channel from the firmware.
This procedure requires sending BASE_DISCOVER_AGENT using func_id and
shmem region for each
registered agent and discover it's agent_id. This operation requires all
memory reserved for the agents
to be mapped and then unmapped which can cause boot time increasing. So
if boot time is important it is
preferable to provide agent_in in the device-tree.
There are some cases when Xen may not have an access to other agent
memory, including scmi shmem due to
a very strict FFI configuration. In this case agent_id should be
provided inside xem,scmi-secondary-agents parameter.
```
What do you think about this approach?
I will add information about all point into the docs and will try to
make it as clear as possible.
>>>> +/{
>>>> + scmi_shm_1: sram@47ff1000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_2: sram@47ff2000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_3: sram@47ff3000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_3: sram@47ff4000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>>>> + };
>>> Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
>>> Or are under /firmware? Or are they under /chosen?
>>>
>>> I take they are under the top level node together with scmi_shm_0?
>>>
>>> Can you please also clarify in the document as well?
>>>
>>>
>> all these nodes are on the top level of the device-tree. But there is no
>> specific place for them.
>>
>> They could be subnodes to some memory-region for example. I will clarify
>> this.
>>
>>>> +}
>>>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
>>>> index 8e50f6b7c7..bc3c64d6ec 100644
>>>> --- a/docs/misc/xen-command-line.pandoc
>>>> +++ b/docs/misc/xen-command-line.pandoc
>>>> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be disabled for Dom0/hwdom and
>>>> SCMI nodes removed from Dom0/hwdom device tree.
>>>> (for example, thin Dom0 with Driver domain use-case).
>>>>
>>>> +### dom0_scmi_agent_id (ARM)
>>>> +> `= <integer>`
>>>> +
>>>> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and allows to
>>>> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI agent id.
>>>> +The SCMI will be disabled for Dom0 if this option is not specified
>>>> +(for example, thin Dom0 or dom0less use-cases).
>>>> +The agent ids of domains existing on a single host must be unique.
>>>> +
>>>> ### dtuart (ARM)
>>>> > `= path [:options]`
>>>>
>>>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>>>> index 28ba9eb787..7712f53cd4 100644
>>>> --- a/tools/libs/light/libxl_arm.c
>>>> +++ b/tools/libs/light/libxl_arm.c
>>>> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>>> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
>>>> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>>> break;
>>>> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
>>>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>>> + config->arch.arm_sci_agent_id = d_config->b_info.arch_arm.arm_sci.agent_id;
>>>> + break;
>>>> default:
>>>> LOG(ERROR, "Unknown ARM_SCI type %d",
>>>> d_config->b_info.arch_arm.arm_sci.type);
>>>> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
>>>> index aa2190ab5b..11e31ce786 100644
>>>> --- a/tools/libs/light/libxl_types.idl
>>>> +++ b/tools/libs/light/libxl_types.idl
>>>> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
>>>>
>>>> libxl_arm_sci_type = Enumeration("arm_sci_type", [
>>>> (0, "none"),
>>>> - (1, "scmi_smc")
>>>> + (1, "scmi_smc"),
>>>> + (2, "scmi_smc_multiagent")
>>>> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
>>>>
>>>> libxl_arm_sci = Struct("arm_sci", [
>>>> ("type", libxl_arm_sci_type),
>>>> + ("agent_id", uint8)
>>>> ])
>>>>
>>>> libxl_rdm_reserve = Struct("rdm_reserve", [
>>>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>>>> index bd22be9d33..81aa3797e3 100644
>>>> --- a/tools/xl/xl_parse.c
>>>> +++ b/tools/xl/xl_parse.c
>>>> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg, libxl_arm_sci *arm_sci,
>>>> }
>>>> }
>>>>
>>>> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
>>>> + unsigned long val = parse_ulong(oparg);
>>>> +
>>>> + if (!val || val > 255) {
>>>> + fprintf(stderr, "An invalid ARM_SCI agent_id specified (%lu). Valid range [1..255]\n",
>>>> + val);
>>>> + ret = ERROR_INVAL;
>>>> + goto parse_error;
>>>> + }
>>>> + arm_sci->agent_id = val;
>>>> + }
>>>> +
>>>> ptr = strtok(NULL, ",");
>>>> }
>>>>
>>>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>>>> index 0a00f03a25..43d21eb889 100644
>>>> --- a/xen/arch/arm/dom0less-build.c
>>>> +++ b/xen/arch/arm/dom0less-build.c
>>>> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node *node,
>>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>>>> else if ( !strcmp(sci_type, "scmi_smc") )
>>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>>> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
>>>> + {
>>>> + uint32_t agent_id = 0;
>>>> +
>>>> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id) ||
>>>> + !agent_id )
>>> shouldn't we check that agent_id <= 255 ?
>> I see no limitation about max agent_id in DEN0056E document, it's uint32_t.
>>
>>>> + return -EINVAL;
>>>> +
>>>> + d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>>> + d_cfg->arch.arm_sci_agent_id = agent_id;
>>>> + }
>>>> else
>>>> {
>>>> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain %s\n",
>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>> index 36d28b52a4..0c9274a2b3 100644
>>>> --- a/xen/arch/arm/domain_build.c
>>>> +++ b/xen/arch/arm/domain_build.c
>>>> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-start") ||
>>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-size") ||
>>>> dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-size") ||
>>>> - dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver"))
>>>> + dt_property_name_is_equal(prop, "linux,uefi-mmap-desc-ver") ||
>>>> + dt_property_name_is_equal(prop, "xen,scmi-secondary-agents") )
>>>> continue;
>>>>
>>>> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") ) diff --git a/xen/arch/arm/firmware/Kconfig
>>>> b/xen/arch/arm/firmware/Kconfig index 5c5f0880c4..6b051c8ada 100644
>>>> --- a/xen/arch/arm/firmware/Kconfig +++
>>>> b/xen/arch/arm/firmware/Kconfig @@ -29,6 +29,17 @@ config SCMI_SMC
>>>> driver domain. Use with EL3 firmware which supports only single SCMI
>>>> OSPM agent. +config SCMI_SMC_MA + bool "Enable ARM SCMI SMC multi-agent driver"
>>>> + select ARM_SCI
>>>> + help
>>>> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from Domains
>>>> + to EL3 firmware (TF-A) which supports multi-agent feature.
>>>> + This feature allows to enable SCMI per Domain using unique SCMI agent_id,
>>>> + so Domain is identified by EL3 firmware as an SCMI Agent and can access
>>>> + allowed platform resources through dedicated SMC/HVC Shared memory based
>>>> + transport.
>>>> +
>>>> endchoice
>>>>
>>>> endmenu
>>>> diff --git a/xen/arch/arm/firmware/Makefile b/xen/arch/arm/firmware/Makefile
>>>> index 71bdefc24a..37927e690e 100644
>>>> --- a/xen/arch/arm/firmware/Makefile
>>>> +++ b/xen/arch/arm/firmware/Makefile
>>>> @@ -1,2 +1,3 @@
>>>> obj-$(CONFIG_ARM_SCI) += sci.o
>>>> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
>>>> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
>>>> diff --git a/xen/arch/arm/firmware/scmi-proto.h b/xen/arch/arm/firmware/scmi-proto.h
>>>> new file mode 100644
>>>> index 0000000000..3f4b9c5d6b
>>>> --- /dev/null
>>>> +++ b/xen/arch/arm/firmware/scmi-proto.h
>>>> @@ -0,0 +1,164 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * Arm System Control and Management Interface definitions
>>>> + * Version 3.0 (DEN0056C)
>>>> + *
>>>> + * Copyright (c) 2024 EPAM Systems
>>>> + */
>>>> +
>>>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>>>> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>>> NIT: ARM_FIRMWARE_SCMI_PROTO_H
>> +
>>>> +#include <xen/stdint.h>
>>>> +
>>>> +#define SCMI_SHORT_NAME_MAX_SIZE 16
>>>> +
>>>> +/* SCMI status codes. See section 4.1.4 */
>>>> +#define SCMI_SUCCESS 0
>>>> +#define SCMI_NOT_SUPPORTED (-1)
>>>> +#define SCMI_INVALID_PARAMETERS (-2)
>>>> +#define SCMI_DENIED (-3)
>>>> +#define SCMI_NOT_FOUND (-4)
>>>> +#define SCMI_OUT_OF_RANGE (-5)
>>>> +#define SCMI_BUSY (-6)
>>>> +#define SCMI_COMMS_ERROR (-7)
>>>> +#define SCMI_GENERIC_ERROR (-8)
>>>> +#define SCMI_HARDWARE_ERROR (-9)
>>>> +#define SCMI_PROTOCOL_ERROR (-10)
>>>> +
>>>> +/* Protocol IDs */
>>>> +#define SCMI_BASE_PROTOCOL 0x10
>>>> +
>>>> +/* Base protocol message IDs */
>>>> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
>>>> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
>>>> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
>>>> +#define SCMI_BASE_DISCOVER_AGENT 0x7
>>>> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
>>>> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
>>>> +
>>>> +typedef struct scmi_msg_header {
>>>> + uint8_t id;
>>>> + uint8_t type;
>>>> + uint8_t protocol;
>>>> + uint32_t status;
>>>> +} scmi_msg_header_t;
>>>> +
>>>> +/* Table 2 Message header format */
>>>> +#define SCMI_HDR_ID GENMASK(7, 0)
>>>> +#define SCMI_HDR_TYPE GENMASK(9, 8)
>>>> +#define SCMI_HDR_PROTO GENMASK(17, 10)
>>>> +
>>>> +#define SCMI_FIELD_GET(_mask, _reg) \
>>>> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
>>>> +#define SCMI_FIELD_PREP(_mask, _val) \
>>>> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
>>>> +
>>>> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
>>>> +{
>>>> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
>>>> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
>>>> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
>>>> +}
>>>> +
>>>> +static inline void unpack_scmi_header(uint32_t msg_hdr, scmi_msg_header_t *hdr)
>>>> +{
>>>> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
>>>> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
>>>> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
>>>> +}
>>>> +
>>>> +static inline int scmi_to_xen_errno(int scmi_status)
>>>> +{
>>>> + if ( scmi_status == SCMI_SUCCESS )
>>>> + return 0;
>>>> +
>>>> + switch ( scmi_status )
>>>> + {
>>>> + case SCMI_NOT_SUPPORTED:
>>>> + return -EOPNOTSUPP;
>>>> + case SCMI_INVALID_PARAMETERS:
>>>> + return -EINVAL;
>>>> + case SCMI_DENIED:
>>>> + return -EACCES;
>>>> + case SCMI_NOT_FOUND:
>>>> + return -ENOENT;
>>>> + case SCMI_OUT_OF_RANGE:
>>>> + return -ERANGE;
>>>> + case SCMI_BUSY:
>>>> + return -EBUSY;
>>>> + case SCMI_COMMS_ERROR:
>>>> + return -ENOTCONN;
>>>> + case SCMI_GENERIC_ERROR:
>>>> + return -EIO;
>>>> + case SCMI_HARDWARE_ERROR:
>>>> + return -ENXIO;
>>>> + case SCMI_PROTOCOL_ERROR:
>>>> + return -EBADMSG;
>>>> + default:
>>>> + return -EINVAL;
>>>> + }
>>>> +}
>>>> +
>>>> +/* PROTOCOL_VERSION */
>>>> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
>>>> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
>>>> +
>>>> +struct scmi_msg_prot_version_p2a {
>>>> + uint32_t version;
>>>> +} __packed;
>>>> +
>>>> +/* BASE PROTOCOL_ATTRIBUTES */
>>>> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
>>>> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
>>>> +
>>>> +struct scmi_msg_base_attributes_p2a {
>>>> + uint32_t attributes;
>>>> +} __packed;
>>>> +
>>>> +/*
>>>> + * BASE_DISCOVER_AGENT
>>>> + */
>>>> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
>>>> +
>>>> +struct scmi_msg_base_discover_agent_a2p {
>>>> + uint32_t agent_id;
>>>> +} __packed;
>>>> +
>>>> +struct scmi_msg_base_discover_agent_p2a {
>>>> + uint32_t agent_id;
>>>> + char name[SCMI_SHORT_NAME_MAX_SIZE];
>>>> +} __packed;
>>>> +
>>>> +/*
>>>> + * BASE_SET_DEVICE_PERMISSIONS
>>>> + */
>>>> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
>>>> +
>>>> +struct scmi_msg_base_set_device_permissions_a2p {
>>>> + uint32_t agent_id;
>>>> + uint32_t device_id;
>>>> + uint32_t flags;
>>>> +} __packed;
>>>> +
>>>> +/*
>>>> + * BASE_RESET_AGENT_CONFIGURATION
>>>> + */
>>>> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
>>>> +
>>>> +struct scmi_msg_base_reset_agent_cfg_a2p {
>>>> + uint32_t agent_id;
>>>> + uint32_t flags;
>>>> +} __packed;
>>>> +
>>>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
>>>> +
>>>> +/*
>>>> + * Local variables:
>>>> + * mode: C
>>>> + * c-file-style: "BSD"
>>>> + * c-basic-offset: 4
>>>> + * tab-width: 4
>>>> + * indent-tabs-mode: nil
>>>> + * End:
>>>> + */
>>>> diff --git a/xen/arch/arm/firmware/scmi-shmem.c b/xen/arch/arm/firmware/scmi-shmem.c
>>>> new file mode 100644
>>>> index 0000000000..dd613ee0b5
>>>> --- /dev/null
>>>> +++ b/xen/arch/arm/firmware/scmi-shmem.c
>>>> @@ -0,0 +1,173 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>>>> + *
>>>> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>> + * Copyright (c) 2025 EPAM Systems
>>>> + */
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +
>>>> +#include <asm/io.h>
>>>> +#include <xen/err.h>
>>>> +
>>>> +#include "scmi-proto.h"
>>>> +#include "scmi-shmem.h"
>>> This code is written more generically than the description implies. If
>>> we only want to make SMC calls to TF-A on EL3 and exchange data with it
>>> over shared memory, then I think:
>>> - we don't need the __iomem tag, as there is no MMIO
>>> - we only need a DMB, not a DSB (readl and writel imply DSB, use only
>>> readl_relaxed and writel_relaxed)
>>>
>>> On the other hand, if we also want to handle the case where the SCMI
>>> server could be on a separate co-processor, then what this code is doing
>>> is not sufficient because we also need a dcache flush, in addition to
>>> the DSB.
>>>
>>> Bertrand, can you double-check?
>>>
>>>
>>>> +/*
>>>> + * Copy data from IO memory space to "real" memory space.
>>>> + */
>>>> +static void __memcpy_fromio(void *to, const volatile void __iomem *from,
>>>> + size_t count)
>>>> +{
>>>> + while ( count && !IS_ALIGNED((unsigned long)from, 4) )
>>>> + {
>>>> + *(u8 *)to = readb_relaxed(from);
>>>> + from++;
>>>> + to++;
>>>> + count--;
>>>> + }
>>>> +
>>>> + while ( count >= 4 )
>>>> + {
>>>> + *(u32 *)to = readl_relaxed(from);
>>>> + from += 4;
>>>> + to += 4;
>>>> + count -= 4;
>>>> + }
>>>> +
>>>> + while ( count )
>>>> + {
>>>> + *(u8 *)to = readb_relaxed(from);
>>>> + from++;
>>>> + to++;
>>>> + count--;
>>>> + }
>>>> +}
>>>> +
>>>> +/*
>>>> + * Copy data from "real" memory space to IO memory space.
>>>> + */
>>>> +static void __memcpy_toio(volatile void __iomem *to, const void *from,
>>>> + size_t count)
>>>> +{
>>>> + while ( count && !IS_ALIGNED((unsigned long)to, 4) )
>>>> + {
>>>> + writeb_relaxed(*(u8 *)from, to);
>>>> + from++;
>>>> + to++;
>>>> + count--;
>>>> + }
>>>> +
>>>> + while ( count >= 4 )
>>>> + {
>>>> + writel_relaxed(*(u32 *)from, to);
>>>> + from += 4;
>>>> + to += 4;
>>>> + count -= 4;
>>>> + }
>>>> +
>>>> + while ( count )
>>>> + {
>>>> + writeb_relaxed(*(u8 *)from, to);
>>>> + from++;
>>>> + to++;
>>>> + count--;
>>>> + }
>>>> +}
>>> I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
>>> we use a simple memcpy?
>> This approach was used because we're trying to access shared memory
>> between two independent systems:
>>
>> Arm-TF and Xen in our case which places some chunks of data to the same
>> memory. And, according to the [0]
>>
>> ```
>>
>> Some devices (such as framebuffers) would like to use larger transfers than
>> 8 bytes at a time. For these devices, the memcpy_toio(),
>> memcpy_fromio() and memset_io() functions are
>> provided. Do not use memset or memcpy on IO addresses; they are not
>> guaranteed to copy data in order.
>>
>> ```
>>
>> Also, the same approach was used by Arm team when introducing scmi
>> driver to the Linux kernel [1]
>>
>>
>> [0]:https://www.kernel.org/doc/Documentation/driver-api/device-io.rst
>>
>> [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
> Keep in mind that [0] refers specifically to access to MMIO regions. I
> assume that the SCMI shared buffers are on normal memory? Regarding [1],
> it makes sense if Linux is trying to support shared memory over MMIO.
>
> Looking at one of your replies below, I am guessing the memory buffers
> are actually in normal memory but the issue is that TF-A is mapping them
> as uncacheable. Is that correct?
>
> In that case, I still don't understand why a simple memcpy would not be
> sufficient. Can you check?
>
> If yes, then for now I would just simplify it down to memcpy. When
> someone adds support for an SCMI server elsewhere we could look into
> adding a more sophisticated memcpy and we can look at the details at
> that point in time. Specifically, I am not convinced that memcpy_toio
> and memcpy_fromio would work if the SCMI server is on a separate
> non-coherent microcontroller.
>
According to the TF-A implementation SCMI memory
is mapped with the flags: MT_DEVICE (like for stm32mp1) or
MT_NON_CACHEABLE (for rpi3)
So probably you're right. I will check with simple memcpy.
>>>> +static inline int
>>>> +shmem_channel_is_free(const volatile struct scmi_shared_mem __iomem *shmem)
>>>> +{
>>>> + return (readl(&shmem->channel_status) &
>>>> + SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE) ? 0 : -EBUSY;
>>>> +}
>>>> +
>>>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>>>> + scmi_msg_header_t *hdr, void *data, int len)
>>>> +{
>>>> + int ret;
>>>> +
>>>> + if ( (len + sizeof(shmem->msg_header)) > SCMI_SHMEM_MAPPED_SIZE )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: Wrong size of smc message. Data is invalid\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + ret = shmem_channel_is_free(shmem);
>>>> + if ( ret )
>>>> + return ret;
>>>> +
>>>> + writel_relaxed(0x0, &shmem->channel_status);
>>>> + /* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
>>>> + writel_relaxed(0x0, &shmem->flags);
>>>> + writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
>>>> + writel(pack_scmi_header(hdr), &shmem->msg_header);
>>>> +
>>>> + if ( len > 0 && data )
>>>> + __memcpy_toio(shmem->msg_payload, data, len);
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>>>> + scmi_msg_header_t *hdr, void *data, int len)
>>>> +{
>>>> + int recv_len;
>>>> + int ret;
>>>> + int pad = sizeof(hdr->status);
>>>> +
>>>> + if ( len >= SCMI_SHMEM_MAPPED_SIZE - sizeof(shmem) )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: Wrong size of input smc message. Data may be invalid\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + ret = shmem_channel_is_free(shmem);
>>>> + if ( ret )
>>>> + return ret;
>>>> +
>>>> + recv_len = readl(&shmem->length) - sizeof(shmem->msg_header);
>>>> +
>>>> + if ( recv_len < 0 )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: Wrong size of smc message. Data may be invalid\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + unpack_scmi_header(readl(&shmem->msg_header), hdr);
>>>> +
>>>> + hdr->status = readl(&shmem->msg_payload);
>>>> + recv_len = recv_len > pad ? recv_len - pad : 0;
>>>> +
>>>> + ret = scmi_to_xen_errno(hdr->status);
>>>> + if ( ret )
>>>> + {
>>>> + printk(XENLOG_DEBUG "scmi: Error received: %d\n", ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + if ( recv_len > len )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: Not enough buffer for message %d, expecting %d\n",
>>>> + recv_len, len);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if ( recv_len > 0 )
>>>> + __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +/*
>>>> + * Local variables:
>>>> + * mode: C
>>>> + * c-file-style: "BSD"
>>>> + * c-basic-offset: 4
>>>> + * tab-width: 4
>>>> + * indent-tabs-mode: nil
>>>> + * End:
>>>> + */
>>>> diff --git a/xen/arch/arm/firmware/scmi-shmem.h b/xen/arch/arm/firmware/scmi-shmem.h
>>>> new file mode 100644
>>>> index 0000000000..2f8e23ff76
>>>> --- /dev/null
>>>> +++ b/xen/arch/arm/firmware/scmi-shmem.h
>>>> @@ -0,0 +1,45 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * Arm System Control and Management Interface definitions
>>>> + * Version 3.0 (DEN0056C)
>>>> + * Shared Memory based Transport
>>>> + *
>>>> + * Copyright (c) 2024 EPAM Systems
>>>> + */
>>>> +
>>>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
>>>> +#define XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_
>>> NIT: ARM_FIRMWARE_SCMI_SHMEM_H
>>>
>> +
>>>> +#include <xen/stdint.h>
>>>> +
>>>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE BIT(0, UL)
>>>> +#define SCMI_SHMEM_CHAN_STAT_CHANNEL_ERROR BIT(1, UL)
>>>> +
>>>> +struct scmi_shared_mem {
>>>> + uint32_t reserved;
>>>> + uint32_t channel_status;
>>>> + uint32_t reserved1[2];
>>>> + uint32_t flags;
>>>> + uint32_t length;
>>>> + uint32_t msg_header;
>>>> + uint8_t msg_payload[];
>>>> +};
>>>> +
>>>> +#define SCMI_SHMEM_MAPPED_SIZE PAGE_SIZE
>>>> +
>>>> +int shmem_put_message(volatile struct scmi_shared_mem __iomem *shmem,
>>>> + scmi_msg_header_t *hdr, void *data, int len);
>>>> +
>>>> +int shmem_get_response(const volatile struct scmi_shared_mem __iomem *shmem,
>>>> + scmi_msg_header_t *hdr, void *data, int len);
>>>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_SHMEM_H_ */
>>>> +
>>>> +/*
>>>> + * Local variables:
>>>> + * mode: C
>>>> + * c-file-style: "BSD"
>>>> + * c-basic-offset: 4
>>>> + * tab-width: 4
>>>> + * indent-tabs-mode: nil
>>>> + * End:
>>>> + */
>>>> diff --git a/xen/arch/arm/firmware/scmi-smc-multiagent.c b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>>>> new file mode 100644
>>>> index 0000000000..e023bca3a1
>>>> --- /dev/null
>>>> +++ b/xen/arch/arm/firmware/scmi-smc-multiagent.c
>>>> @@ -0,0 +1,860 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>>>> + *
>>>> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>> + * Copyright (c) 2025 EPAM Systems
>>>> + */
>>>> +
>>>> +#include <xen/acpi.h>
>>>> +
>>>> +#include <xen/device_tree.h>
>>>> +#include <xen/init.h>
>>>> +#include <xen/iocap.h>
>>>> +#include <xen/err.h>
>>>> +#include <xen/libfdt/libfdt.h>
>>>> +#include <xen/param.h>
>>>> +#include <xen/sched.h>
>>>> +#include <xen/vmap.h>
>>>> +
>>>> +#include <asm/firmware/sci.h>
>>>> +#include <asm/smccc.h>
>>>> +
>>>> +#include "scmi-proto.h"
>>>> +#include "scmi-shmem.h"
>>>> +
>>>> +#define SCMI_AGENT_ID_INVALID 0xFF
>>>> +
>>>> +static uint8_t __initdata opt_dom0_scmi_agent_id = SCMI_AGENT_ID_INVALID;
>>>> +integer_param("dom0_scmi_agent_id", opt_dom0_scmi_agent_id);
>>>> +
>>>> +#define SCMI_SECONDARY_AGENTS "xen,scmi-secondary-agents"
>>>> +
>>>> +#define HYP_CHANNEL 0x0
>>>> +
>>>> +struct scmi_channel {
>>>> + uint32_t agent_id;
>>>> + uint32_t func_id;
>>>> + domid_t domain_id;
>>>> + uint64_t paddr;
>>>> + uint64_t len;
>>>> + struct scmi_shared_mem __iomem *shmem;
>>>> + spinlock_t lock;
>>>> + struct list_head list;
>>>> +};
>>>> +
>>>> +struct scmi_data {
>>>> + struct list_head channel_list;
>>>> + spinlock_t channel_list_lock;
>>>> + uint32_t func_id;
>>>> + bool initialized;
>>>> + uint32_t shmem_phandle;
>>>> + struct dt_device_node *dt_dev;
>>>> +};
>>>> +
>>>> +static struct scmi_data scmi_data;
>>>> +
>>>> +static int send_smc_message(struct scmi_channel *chan_info,
>>>> + scmi_msg_header_t *hdr, void *data, int len)
>>>> +{
>>>> + struct arm_smccc_res resp;
>>>> + int ret;
>>>> +
>>>> + ret = shmem_put_message(chan_info->shmem, hdr, data, len);
>>>> + if ( ret )
>>>> + return ret;
>>>> +
>>>> + arm_smccc_1_1_smc(chan_info->func_id, 0, 0, 0, 0, 0, 0, 0, &resp);
>>>> +
>>>> + if ( resp.a0 )
>>>> + return -EOPNOTSUPP;
>>> Why if repo.a0 != 0 then we assume -EOPNOTSUPP? Is this part of the SCMI
>>> specification?
>> Please see 7.1 of [1]. The following codes are supported :
>>
>> 0 for success
>>
>> -1 as not_supported
>>
>> -2 as not_required
>>
>> -3 as invalid_parameter.
>>
>> For all non-zero results we assume that this call is not supported by
>> system or has different revision
>>
>> which we also doesn't support.
> For -1 and -2 make sense. I am less sure about converting -3 into
> -EOPNOTSUPP. But I'll trust your judgment.
>
I agree, I will return -EINVAL for -3 return code.
>> [1] DEN0028
>> https://documentation-service.arm.com/static/5f8edaeff86e16515cdbe4c6
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int do_smc_xfer(struct scmi_channel *chan_info, scmi_msg_header_t *hdr,
>>>> + void *tx_data, int tx_size, void *rx_data, int rx_size)
>>>> +{
>>>> + int ret = 0;
>>>> +
>>>> + ASSERT(chan_info && chan_info->shmem);
>>>> +
>>>> + if ( !hdr )
>>>> + return -EINVAL;
>>>> +
>>>> + spin_lock(&chan_info->lock);
>>>> +
>>>> + printk(XENLOG_DEBUG
>>>> + "scmi: agent_id = %d msg_id = %x type = %d, proto = %x\n",
>>>> + chan_info->agent_id, hdr->id, hdr->type, hdr->protocol);
>>>> +
>>>> + ret = send_smc_message(chan_info, hdr, tx_data, tx_size);
>>>> + if ( ret )
>>>> + goto clean;
>>>> +
>>>> + ret = shmem_get_response(chan_info->shmem, hdr, rx_data, rx_size);
>>>> +
>>>> +clean:
>>>> + printk(XENLOG_DEBUG
>>>> + "scmi: get smc response agent_id = %d msg_id = %x proto = %x res=%d\n",
>>>> + chan_info->agent_id, hdr->id, hdr->protocol, ret);
>>>> +
>>>> + spin_unlock(&chan_info->lock);
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +static struct scmi_channel *get_channel_by_id(uint32_t agent_id)
>>>> +{
>>>> + struct scmi_channel *curr;
>>>> + bool found = false;
>>>> +
>>>> + spin_lock(&scmi_data.channel_list_lock);
>>>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>>>> + {
>>>> + if ( curr->agent_id == agent_id )
>>>> + {
>>>> + found = true;
>>>> + break;
>>>> + }
>>>> + }
>>>> +
>>>> + spin_unlock(&scmi_data.channel_list_lock);
>>>> + if ( found )
>>>> + return curr;
>>>> +
>>>> + return NULL;
>>>> +}
>>>> +
>>>> +static struct scmi_channel *acquire_scmi_channel(struct domain *d,
>>>> + uint32_t agent_id)
>>>> +{
>>>> + struct scmi_channel *curr;
>>>> + struct scmi_channel *ret = ERR_PTR(-ENOENT);
>>>> +
>>>> + spin_lock(&scmi_data.channel_list_lock);
>>>> + list_for_each_entry(curr, &scmi_data.channel_list, list)
>>>> + {
>>>> + if ( curr->agent_id == agent_id )
>>>> + {
>>>> + if ( curr->domain_id != DOMID_INVALID )
>>>> + {
>>>> + ret = ERR_PTR(-EEXIST);
>>>> + break;
>>>> + }
>>>> +
>>>> + curr->domain_id = d->domain_id;
>>>> + ret = curr;
>>>> + break;
>>>> + }
>>>> + }
>>>> +
>>>> + spin_unlock(&scmi_data.channel_list_lock);
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +static void relinquish_scmi_channel(struct scmi_channel *channel)
>>>> +{
>>>> + ASSERT(channel != NULL);
>>>> +
>>>> + spin_lock(&scmi_data.channel_list_lock);
>>>> + channel->domain_id = DOMID_INVALID;
>>>> + spin_unlock(&scmi_data.channel_list_lock);
>>>> +}
>>>> +
>>>> +static int map_channel_memory(struct scmi_channel *channel)
>>>> +{
>>>> + ASSERT(channel && channel->paddr);
>>>> + channel->shmem = ioremap_nocache(channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
>>> ioremap is for MMIO, if these shared memory channels are on DDR, then it
>>> would not be the right call. Are the "arm,scmi-shmem" address ranges
>>> part of the memory node ranges? Or are they completely separate?
>>>
>>> Also, why nocache? Wouldn't we want ioremap_cache?
>>>
>> Currently, they are separate nodes but could potentially be part of the
>> "memory-region".
>>
>> From what I see in the Linux kernel, devm_ioremap is being used to map
>> the memory.
>>
>> Could there be something more to this approach?
>>
>> [0]:https://git.iliana.fyi/linux/patch/?id=1dc6558062dadfabd2fb3bd885fa6e92ec7196f2
> I am guessing that TF-A is mapping memory as uncacheable so we should do
> the same in Xen.
Agree.
>>>> + if ( !channel->shmem )
>>>> + return -ENOMEM;
>>>> +
>>>> + channel->shmem->channel_status = SCMI_SHMEM_CHAN_STAT_CHANNEL_FREE;
>>>> + printk(XENLOG_DEBUG "scmi: Got shmem %lx after vmap %p\n", channel->paddr,
>>>> + channel->shmem);
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static void unmap_channel_memory(struct scmi_channel *channel)
>>>> +{
>>>> + ASSERT(channel && channel->shmem);
>>>> + iounmap(channel->shmem);
>>>> + channel->shmem = NULL;
>>>> +}
>>>> +
>>>> +static struct scmi_channel *smc_create_channel(uint32_t agent_id,
>>>> + uint32_t func_id, uint64_t addr)
>>>> +{
>>>> + struct scmi_channel *channel;
>>>> +
>>>> + channel = get_channel_by_id(agent_id);
>>>> + if ( channel )
>>>> + return ERR_PTR(EEXIST);
>>>> +
>>>> + channel = xmalloc(struct scmi_channel);
>>>> + if ( !channel )
>>>> + return ERR_PTR(ENOMEM);
>>>> +
>>>> + spin_lock_init(&channel->lock);
>>>> + channel->agent_id = agent_id;
>>>> + channel->func_id = func_id;
>>>> + channel->domain_id = DOMID_INVALID;
>>>> + channel->shmem = NULL;
>>>> + channel->paddr = addr;
>>>> + list_add_tail(&channel->list, &scmi_data.channel_list);
>>>> + return channel;
>>>> +}
>>>> +
>>>> +static void free_channel_list(void)
>>>> +{
>>>> + struct scmi_channel *curr, *_curr;
>>>> +
>>>> + list_for_each_entry_safe(curr, _curr, &scmi_data.channel_list, list)
>>>> + {
>>>> + list_del(&curr->list);
>>>> + xfree(curr);
>>>> + }
>>>> +}
>>>> +
>>>> +static int __init
>>>> +scmi_dt_read_hyp_channel_addr(struct dt_device_node *scmi_node, u64 *addr,
>>>> + u64 *size)
>>>> +{
>>>> + struct dt_device_node *shmem_node;
>>>> + const __be32 *prop;
>>>> +
>>>> + prop = dt_get_property(scmi_node, "shmem", NULL);
>>>> + if ( !prop )
>>>> + return -EINVAL;
>>>> +
>>>> + shmem_node = dt_find_node_by_phandle(be32_to_cpup(prop));
>>>> + if ( IS_ERR_OR_NULL(shmem_node) )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: Device tree error, can't parse reserved memory %ld\n",
>>>> + PTR_ERR(shmem_node));
>>>> + return PTR_ERR(shmem_node);
>>>> + }
>>>> +
>>>> + return dt_device_get_address(shmem_node, 0, addr, size);
>>>> +}
>>>> +
>>>> +/*
>>>> + * Handle Dom0 SCMI specific DT nodes
>>>> + *
>>>> + * Make a decision on copying SCMI specific nodes into Dom0 device tree.
>>>> + * For SCMI multi-agent case:
>>>> + * - shmem nodes will not be copied and generated instead if SCMI
>>>> + * is enabled for Dom0
>>>> + * - scmi node will be copied if SCMI is enabled for Dom0
>>>> + */
>>>> +static bool scmi_dt_handle_node(struct domain *d, struct dt_device_node *node)
>>>> +{
>>>> + static const struct dt_device_match skip_matches[] __initconst = {
>>>> + DT_MATCH_COMPATIBLE("arm,scmi-shmem"),
>>>> + { /* sentinel */ },
>>>> + };
>>>> + static const struct dt_device_match scmi_matches[] __initconst = {
>>>> + DT_MATCH_PATH("/firmware/scmi"),
>>>> + { /* sentinel */ },
>>>> + };
>>>> +
>>>> + if ( !scmi_data.initialized )
>>>> + return false;
>>>> +
>>>> + /* always drop shmem */
>>>> + if ( dt_match_node(skip_matches, node) )
>>>> + {
>>>> + dt_dprintk(" Skip scmi shmem\n");
>>>> + return true;
>>>> + }
>>>> +
>>>> + /* drop scmi if not enabled */
>>>> + if ( dt_match_node(scmi_matches, node) && !sci_domain_is_enabled(d) )
>>>> + {
>>>> + dt_dprintk(" Skip scmi node\n");
>>>> + return true;
>>>> + }
>>>> +
>>>> + return false;
>>>> +}
>>>> +
>>>> +/*
>>>> + * Finalize Dom0 SCMI specific DT nodes
>>>> + *
>>>> + * if SCMI is enabled for Dom0:
>>>> + * - generate shmem node
>>>> + * - map SCMI shmem MMIO into Dom0
>>>> + */
>>>> +static int scmi_dt_finalize(struct domain *d, void *fdt)
>>>> +{
>>>> + __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
>>>> + struct scmi_channel *channel;
>>>> + int nodeoffset;
>>>> + __be32 *cells;
>>>> + __be32 val;
>>>> + char buf[64];
>>>> + int res, rc;
>>>> +
>>>> + if ( !sci_domain_is_enabled(d) )
>>>> + return 0;
>>>> +
>>>> + channel = d->arch.sci_data;
>>>> +
>>>> + /*
>>>> + * Replace "arm,smc-id" with proper value assigned for Dom0 SCMI channel
>>>> + */
>>>> + nodeoffset = fdt_node_offset_by_compatible(fdt, -1, "arm,scmi-smc");
>>>> + if ( nodeoffset < 0 )
>>>> + return -ENODEV;
>>>> +
>>>> + cells = (__be32 *)&val;
>>>> + dt_set_cell(&cells, 1, channel->func_id);
>>>> + res = fdt_setprop_inplace(fdt, nodeoffset, "arm,smc-id", &val, sizeof(val));
>>>> + if ( res )
>>>> + return -EINVAL;
>>>> +
>>> Are you sure it is worth to go through all this trouble to modify FDT in
>>> place when we could simply generate the DT node from scratch like we do
>>> for example for the GIC? This seems to be more error prone as well. Is
>>> generating it from scratch is really difficult? If it is difficult then OK.
>>>
>> In the last patch [0] of the series, there is a proposal to separate
>>
>> the Xen privileged agent from the Dom0 agent. This eliminates the
>>
>> need to modify the Xen Device Tree Blob (DTB), which is a positive
>> improvement
>>
>> since the Dom0 agent node is already present in the device tree.
>>
>> However, the issue with SCMI node generation lies in the need to handle
>>
>> the list of protocols that are supported across the system.
>>
>> If we want to generate the SCMI node for Dom0, we need to:
>>
>> Copy these protocols from a centralized or predefined source.
>> Set the correct phandle for each subnode within the SCMI node, ensuring
>>
>> accurate representation and functionality.
>> This extra step of managing the protocols and phandles adds
>>
>> complexity but is necessary for ensuring proper support for Dom0.
>>
>> [0]
>> https://lists.xenproject.org/archives/html/xen-devel/2025-05/msg01041.html
> I was only commenting that rather than trying to modify the DT in place
> we could create the node for Dom0 from scratch (artificially), based on
> host DT information as required (fetching data from the host DT as
> required and copying it to the Dom0 DT).
>
The problem is that scmi node, apart from the default parameters has a
list of
supported protocols which may be different for Xen and Dom0 so to
generate Dom0 node
from scratch we need to copy(or even generate) these nodes which will
require complex implementation.
In contrary, if we go with implementation, proposed in the last patch -
then we can have
more cleaner Xen device tree, which will look like this:
```
chosen {
scmi_xen: scmi {
arm,smc-id = <0x82000002>;
shmem= <$shm_0>;
... # no protocols description here
};
firmware {
scmi {
arm,smc-id = <0x82000003>;
shmem= <$shm_1>;
protocol@X{
};
}
};
```
In this case, we don’t provide any protocol information for the Xen
agent and instead set protocols for Dom0.
This ensures that each node has the information it requires.
>>>> + /*
>>>> + * All SCMI shmem nodes should be removed from Dom0 DT at this point, so
>>>> + * the shmem node for Dom0 need to be generated from SCMI channel assigned
>>>> + * to Dom0.
>>>> + * The original SCMI shmem node from platform DT is used by Xen SCMI driver
>>>> + * itself as privileged channel (agent_id=0) to manage other SCMI
>>>> + * agents (domains).
>>>> + */
>>>> + snprintf(buf, sizeof(buf), "scmi-shmem@%lx", channel->paddr);
>>>> +
>>>> + res = fdt_begin_node(fdt, buf);
>>>> + if ( res )
>>>> + return res;
>>>> +
>>>> + res = fdt_property_string(fdt, "compatible", "arm,scmi-shmem");
>>>> + if ( res )
>>>> + return res;
>>>> +
>>>> + cells = ®[0];
>>>> +
>>>> + dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>>>> + channel->paddr, SCMI_SHMEM_MAPPED_SIZE);
>>>> +
>>>> + res = fdt_property(fdt, "reg", reg, sizeof(reg));
>>>> + if ( res )
>>>> + return res;
>>>> +
>>>> + res = fdt_property_cell(fdt, "phandle", scmi_data.shmem_phandle);
>>>> + if ( res )
>>>> + return res;
>>>> +
>>>> + res = fdt_end_node(fdt);
>>>> + if ( res )
>>>> + return res;
>>>> +
>>>> + /*
>>>> + * Map SCMI shmem into Dom0 here as shmem nodes are excluded from
>>>> + * generic Dom0 DT processing
>>>> + */
>>>> + res = iomem_permit_access(d, paddr_to_pfn(channel->paddr),
>>>> + paddr_to_pfn(channel->paddr +
>>>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>>>> + if ( res )
>>>> + return res;
>>>> +
>>>> + res = map_regions_p2mt(d, gaddr_to_gfn(channel->paddr),
>>>> + PFN_UP(SCMI_SHMEM_MAPPED_SIZE),
>>>> + maddr_to_mfn(channel->paddr), p2m_mmio_direct_nc);
>>>> + if ( res )
>>>> + {
>>>> + rc = iomem_deny_access(d, paddr_to_pfn(channel->paddr),
>>>> + paddr_to_pfn(channel->paddr +
>>>> + SCMI_SHMEM_MAPPED_SIZE - 1));
>>>> + if ( rc )
>>>> + printk(XENLOG_ERR "scmi: Unable to deny iomem access , err = %d\n",
>>>> + rc);
>>>> + }
>>>> +
>>>> + return res;
>>>> +}
>>>> +
>>>> +static int scmi_assign_device(uint32_t agent_id, uint32_t device_id,
>>>> + uint32_t flags)
>>>> +{
>>>> + struct scmi_msg_base_set_device_permissions_a2p tx;
>>>> + struct scmi_channel *channel;
>>>> + scmi_msg_header_t hdr;
>>>> + int ret;
>>>> +
>>>> + channel = get_channel_by_id(HYP_CHANNEL);
>>>> + if ( !channel )
>>>> + return -EINVAL;
>>>> +
>>>> + hdr.id = SCMI_BASE_SET_DEVICE_PERMISSIONS;
>>>> + hdr.type = 0;
>>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>>> +
>>>> + tx.agent_id = agent_id;
>>>> + tx.device_id = device_id;
>>>> + tx.flags = flags;
>>>> +
>>>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>>>> + if ( ret == -EOPNOTSUPP )
>>>> + return 0;
>>> Is it actually OK to pretend that everything worked if the return is
>>> -EOPNOTSUPP? I mean that in this case can we assume that the device is
>>> actually assigned anyway? Wouldn't follow up SCMI operations on this
>>> device fail?
>>>
>> I think you right. Will fix in v5
>>>> + return ret;
>>>> +}
>>>> +
>>>> +static int scmi_dt_assign_device(struct domain *d,
>>>> + struct dt_phandle_args *ac_spec)
>>>> +{
>>>> + struct scmi_channel *agent_channel;
>>>> + uint32_t scmi_device_id = ac_spec->args[0];
>>>> + int ret;
>>>> +
>>>> + if ( !d->arch.sci_data )
>>>> + return 0;
>>>> +
>>>> + /* The access-controllers is specified for DT dev, but it's not a SCMI */
>>>> + if ( ac_spec->np != scmi_data.dt_dev )
>>>> + return 0;
>>> I wonder if this should be an error
>>>
>> We don’t expect the DT node access-controller to rely solely on SCMI.
>> Therefore,
>>
>> if a DT node has an access-controller but it’s not associated with SCMI,
>> we should simply
>>
>> ignore that node without raising an error.
>>
>>
>> That’s my understanding of the situation.
> I understand that, but should scmi_dt_assign_device be called in such as
> a case? I didn't go back and look at the original code in writing this
> comment.
This function is part of the SCI interface. I’ve added this check here
to avoid creating an additional function
solely to verify if the access controller node is an SCMI-related node.
Additionally, other potential implementations might use their own
bindings, which could differ from the ones
currently used by SCMI.
My approach is to have a single function that includes all necessary
checks before performing any real work.
>
>>>> + agent_channel = d->arch.sci_data;
>>>> +
>>>> + spin_lock(&agent_channel->lock);
>>>> +
>>>> + ret = scmi_assign_device(agent_channel->agent_id, scmi_device_id,
>>>> + SCMI_BASE_DEVICE_ACCESS_ALLOW);
>>>> + if ( ret )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: could not assign dev for %pd agent:%d dev_id:%u (%d)",
>>>> + d, agent_channel->agent_id, scmi_device_id, ret);
>>>> + }
>>>> +
>>>> + spin_unlock(&agent_channel->lock);
>>>> + return ret;
>>>> +}
>>>> +
>>>> +static __init int collect_agents(struct dt_device_node *scmi_node)
>>>> +{
>>>> + const struct dt_device_node *chosen_node;
>>>> + const __be32 *prop;
>>>> + uint32_t len, i;
>>>> +
>>>> + chosen_node = dt_find_node_by_path("/chosen");
>>>> + if ( !chosen_node )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: chosen node not found\n");
>>>> + return -ENOENT;
>>>> + }
>>>> +
>>>> + prop = dt_get_property(chosen_node, SCMI_SECONDARY_AGENTS, &len);
>>>> + if ( !prop )
>>>> + {
>>>> + printk(XENLOG_WARNING "scmi: No %s property found\n",
>>>> + SCMI_SECONDARY_AGENTS);
>>>> + return -ENODEV;
>>>> + }
>>>> +
>>>> + if ( len % (3 * sizeof(uint32_t)) )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: Invalid length of %s property: %d\n",
>>>> + SCMI_SECONDARY_AGENTS, len);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + for ( i = 0; i < len / (3 * sizeof(uint32_t)); i++ )
>>>> + {
>>>> + uint32_t agent_id = be32_to_cpu(*prop++);
>>>> + uint32_t smc_id = be32_to_cpu(*prop++);
>>>> + uint32_t shmem_phandle = be32_to_cpu(*prop++);
>>>> + struct dt_device_node *node = dt_find_node_by_phandle(shmem_phandle);
>>>> + u64 addr, size;
>>>> + int ret;
>>>> +
>>>> + if ( !node )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: Could not find shmem node for agent %d\n",
>>>> + agent_id);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + ret = dt_device_get_address(node, 0, &addr, &size);
>>>> + if ( ret )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: Could not read shmem address for agent %d: %d",
>>>> + agent_id, ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + ret = PTR_RET(smc_create_channel(agent_id, smc_id, addr));
>>>> + if ( ret )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: Could not create channel for agent %d: %d",
>>>> + agent_id, ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + printk(XENLOG_DEBUG "scmi: Agent %d SMC %X addr %lx\n", agent_id,
>>>> + smc_id, addr);
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int scmi_domain_init(struct domain *d,
>>>> + struct xen_domctl_createdomain *config)
>>>> +{
>>>> + struct scmi_channel *channel;
>>>> + int ret;
>>>> +
>>>> + if ( !scmi_data.initialized )
>>>> + return 0;
>>>> +
>>>> + /*
>>>> + * Special case for Dom0 - the SCMI support is enabled basing on
>>>> + * "dom0_sci_agent_id" Xen command line parameter
>>>> + */
>>>> + if ( is_hardware_domain(d) )
>>>> + {
>>>> + if ( opt_dom0_scmi_agent_id != SCMI_AGENT_ID_INVALID )
>>>> + {
>>>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>>> + config->arch.arm_sci_agent_id = opt_dom0_scmi_agent_id;
>>>> + }
>>>> + else
>>>> + config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>>>> + }
>>>> +
>>>> + if ( config->arch.arm_sci_type == XEN_DOMCTL_CONFIG_ARM_SCI_NONE )
>>>> + return 0;
>>>> +
>>>> + channel = acquire_scmi_channel(d, config->arch.arm_sci_agent_id);
>>>> + if ( IS_ERR(channel) )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: Failed to acquire SCMI channel for agent_id %u: %ld\n",
>>>> + config->arch.arm_sci_agent_id, PTR_ERR(channel));
>>>> + return PTR_ERR(channel);
>>>> + }
>>>> +
>>>> + printk(XENLOG_INFO
>>>> + "scmi: Acquire channel id = 0x%x, domain_id = %d paddr = 0x%lx\n",
>>>> + channel->agent_id, channel->domain_id, channel->paddr);
>>>> +
>>>> + /*
>>>> + * Dom0 (if present) needs to have an access to the guest memory range
>>>> + * to satisfy iomem_access_permitted() check in XEN_DOMCTL_iomem_permission
>>>> + * domctl.
>>> Ideally this should not be needed but I understand we don't have an
>>> easy solution, I think we can go ahead with this for now.
>>>
>>>> + */
>>>> + if ( hardware_domain && !is_hardware_domain(d) )
>>>> + {
>>>> + ret = iomem_permit_access(hardware_domain, paddr_to_pfn(channel->paddr),
>>>> + paddr_to_pfn(channel->paddr + PAGE_SIZE - 1));
>>>> + if ( ret )
>>>> + goto error;
>>>> + }
>>>> +
>>>> + d->arch.sci_data = channel;
>>>> + d->arch.sci_enabled = true;
>>>> +
>>>> + return 0;
>>>> +
>>>> +error:
>>>> + relinquish_scmi_channel(channel);
>>>> + return ret;
>>>> +}
>>>> +
>>>> +int scmi_domain_sanitise_config(struct xen_domctl_createdomain *config)
>>>> +{
>>>> + if ( config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_NONE &&
>>>> + config->arch.arm_sci_type != XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA )
>>>> + {
>>>> + dprintk(XENLOG_INFO, "scmi: Unsupported ARM_SCI type\n");
>>>> + return -EINVAL;
>>>> + }
>>>> + else if ( config->arch.arm_sci_type ==
>>>> + XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA &&
>>>> + config->arch.arm_sci_agent_id == 0 )
>>>> + {
>>>> + dprintk(XENLOG_INFO,
>>>> + "scmi: A zero ARM SCMI agent_id is not supported\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int scmi_relinquish_resources(struct domain *d)
>>>> +{
>>>> + int ret;
>>>> + struct scmi_channel *channel, *agent_channel;
>>>> + scmi_msg_header_t hdr;
>>>> + struct scmi_msg_base_reset_agent_cfg_a2p tx;
>>>> +
>>>> + if ( !d->arch.sci_data )
>>>> + return 0;
>>>> +
>>>> + agent_channel = d->arch.sci_data;
>>>> +
>>>> + spin_lock(&agent_channel->lock);
>>>> + tx.agent_id = agent_channel->agent_id;
>>>> + spin_unlock(&agent_channel->lock);
>>>> +
>>>> + channel = get_channel_by_id(HYP_CHANNEL);
>>>> + if ( !channel )
>>>> + {
>>>> + printk(XENLOG_ERR
>>>> + "scmi: Unable to get Hypervisor scmi channel for domain %d\n",
>>>> + d->domain_id);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + hdr.id = SCMI_BASE_RESET_AGENT_CONFIGURATION;
>>>> + hdr.type = 0;
>>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>>> +
>>>> + tx.flags = 0;
>>>> +
>>>> + ret = do_smc_xfer(channel, &hdr, &tx, sizeof(tx), NULL, 0);
>>>> + if ( ret == -EOPNOTSUPP )
>>>> + return 0;
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +static void scmi_domain_destroy(struct domain *d)
>>>> +{
>>>> + struct scmi_channel *channel;
>>>> +
>>>> + if ( !d->arch.sci_data )
>>>> + return;
>>>> +
>>>> + channel = d->arch.sci_data;
>>>> + spin_lock(&channel->lock);
>>>> +
>>>> + relinquish_scmi_channel(channel);
>>>> + printk(XENLOG_DEBUG "scmi: Free domain %d\n", d->domain_id);
>>>> +
>>>> + d->arch.sci_data = NULL;
>>>> + d->arch.sci_enabled = true;
>>>> +
>>>> + spin_unlock(&channel->lock);
>>>> +}
>>>> +
>>>> +static bool scmi_handle_call(struct cpu_user_regs *regs)
>>>> +{
>>>> + uint32_t fid = (uint32_t)get_user_reg(regs, 0);
>>>> + struct scmi_channel *agent_channel;
>>>> + struct domain *d = current->domain;
>>>> + struct arm_smccc_res resp;
>>>> + bool res = false;
>>>> +
>>>> + if ( !sci_domain_is_enabled(d) )
>>>> + return false;
>>>> +
>>>> + agent_channel = d->arch.sci_data;
>>>> + spin_lock(&agent_channel->lock);
>>>> +
>>>> + if ( agent_channel->func_id != fid )
>>>> + {
>>>> + res = false;
>>>> + goto unlock;
>>>> + }
>>>> +
>>>> + arm_smccc_1_1_smc(fid,
>>>> + get_user_reg(regs, 1),
>>>> + get_user_reg(regs, 2),
>>>> + get_user_reg(regs, 3),
>>>> + get_user_reg(regs, 4),
>>>> + get_user_reg(regs, 5),
>>>> + get_user_reg(regs, 6),
>>>> + get_user_reg(regs, 7),
>>>> + &resp);
>>>> +
>>>> + set_user_reg(regs, 0, resp.a0);
>>>> + set_user_reg(regs, 1, resp.a1);
>>>> + set_user_reg(regs, 2, resp.a2);
>>>> + set_user_reg(regs, 3, resp.a3);
>>>> + res = true;
>>>> +unlock:
>>>> + spin_unlock(&agent_channel->lock);
>>>> +
>>>> + return res;
>>>> +}
>>>> +
>>>> +static const struct sci_mediator_ops scmi_ops = {
>>>> + .domain_init = scmi_domain_init,
>>>> + .domain_destroy = scmi_domain_destroy,
>>>> + .relinquish_resources = scmi_relinquish_resources,
>>>> + .handle_call = scmi_handle_call,
>>>> + .dom0_dt_handle_node = scmi_dt_handle_node,
>>>> + .dom0_dt_finalize = scmi_dt_finalize,
>>>> + .domain_sanitise_config = scmi_domain_sanitise_config,
>>>> + .assign_dt_device = scmi_dt_assign_device,
>>>> +};
>>>> +
>>>> +static int __init scmi_check_smccc_ver(void)
>>>> +{
>>>> + if ( smccc_ver < ARM_SMCCC_VERSION_1_1 )
>>>> + {
>>>> + printk(XENLOG_WARNING
>>>> + "scmi: No SMCCC 1.1 support, SCMI calls forwarding disabled\n");
>>>> + return -ENOSYS;
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static __init int scmi_probe(struct dt_device_node *scmi_node, const void *data)
>>>> +{
>>>> + u64 addr, size;
>>>> + int ret, i;
>>>> + struct scmi_channel *channel, *agent_channel;
>>>> + int n_agents;
>>>> + scmi_msg_header_t hdr;
>>>> + struct scmi_msg_base_attributes_p2a rx;
>>>> +
>>>> + ASSERT(scmi_node != NULL);
>>>> +
>>>> + INIT_LIST_HEAD(&scmi_data.channel_list);
>>>> + spin_lock_init(&scmi_data.channel_list_lock);
>>>> +
>>>> + if ( !acpi_disabled )
>>>> + {
>>>> + printk(XENLOG_WARNING "scmi: is not supported when using ACPI\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + ret = scmi_check_smccc_ver();
>>>> + if ( ret )
>>>> + return ret;
>>>> +
>>>> + if ( !dt_property_read_u32(scmi_node, "arm,smc-id", &scmi_data.func_id) )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: unable to read smc-id from DT\n");
>>>> + return -ENOENT;
>>>> + }
>>>> +
>>>> + /* save shmem phandle and re-use it fro Dom0 DT shmem node */
>>>> + if ( !dt_property_read_u32(scmi_node, "shmem", &scmi_data.shmem_phandle) )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: unable to read shmem phandle from DT\n");
>>>> + return -ENOENT;
>>>> + }
>>>> +
>>>> + ret = scmi_dt_read_hyp_channel_addr(scmi_node, &addr, &size);
>>>> + if ( IS_ERR_VALUE(ret) )
>>>> + return -ENOENT;
>>>> +
>>>> + if ( !IS_ALIGNED(size, SCMI_SHMEM_MAPPED_SIZE) )
>>>> + {
>>>> + printk(XENLOG_ERR "scmi: shmem memory is not aligned\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + scmi_data.dt_dev = scmi_node;
>>>> +
>>>> + channel = smc_create_channel(HYP_CHANNEL, scmi_data.func_id, addr);
>>>> + if ( IS_ERR(channel) )
>>>> + goto out;
>>>> +
>>>> + ret = map_channel_memory(channel);
>>>> + if ( ret )
>>>> + goto out;
>>>> +
>>>> + channel->domain_id = DOMID_XEN;
>>>> +
>>>> + hdr.id = SCMI_BASE_PROTOCOL_ATTIBUTES;
>>>> + hdr.type = 0;
>>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>>> +
>>>> + ret = do_smc_xfer(channel, &hdr, NULL, 0, &rx, sizeof(rx));
>>>> + if ( ret )
>>>> + goto error;
>>>> +
>>>> + n_agents = SCMI_FIELD_GET(SCMI_BASE_ATTR_NUM_AGENT, rx.attributes);
>>>> + printk(XENLOG_DEBUG "scmi: Got agent count %d\n", n_agents);
>>>> +
>>>> + ret = collect_agents(scmi_node);
>>>> + if ( ret )
>>>> + goto error;
>>>> +
>>>> + i = 1;
>>>> +
>>>> + list_for_each_entry(agent_channel, &scmi_data.channel_list, list)
>>>> + {
>>>> + struct scmi_msg_base_discover_agent_p2a da_rx;
>>>> + struct scmi_msg_base_discover_agent_a2p da_tx;
>>>> +
>>>> + ret = map_channel_memory(agent_channel);
>>>> + if ( ret )
>>>> + goto error;
>>>> +
>>>> + hdr.id = SCMI_BASE_DISCOVER_AGENT;
>>>> + hdr.type = 0;
>>>> + hdr.protocol = SCMI_BASE_PROTOCOL;
>>>> +
>>>> + da_tx.agent_id = agent_channel->agent_id;
>>>> +
>>>> + ret = do_smc_xfer(agent_channel, &hdr, &da_tx, sizeof(da_tx), &da_rx,
>>>> + sizeof(da_rx));
>>>> + if ( agent_channel->domain_id != DOMID_XEN )
>>>> + unmap_channel_memory(agent_channel);
>>>> + if ( ret )
>>>> + goto error;
>>>> +
>>>> + printk(XENLOG_DEBUG "id=0x%x name=%s\n", da_rx.agent_id, da_rx.name);
>>>> +
>>>> + agent_channel->agent_id = da_rx.agent_id;
>>> It is OK to set agent_channel->agent_id to the value provided by the
>>> SCMI server, but if we are also taking the agent_channel->agent_id value
>>> from the user via device tree, shouldn't we throw an error if there is a
>>> mismatch?
>>>
>>> Or even better: can we avoid taking the value via device tree to make it
>>> easier to configure?
>>>
>> I plan to drop the last line, as there’s no need to overwrite agent_id.
>>
>> However, we cannot avoid obtaining agent_id from the device tree because,
>>
>> according to section 4.2.2.9 [0], an agent can only be discovered by its
>> agent_id in
>>
>> a privileged domain. Alternatively, each agent can obtain its own ID by
>> providing
>>
>> 0xFFFFFFFF as agent_id and using its own channel.
> That would be great! Anything to reduce the number of required (not
> optional) configuration options.
Please see my comment above. I've marked it with [OM ANSWER] so you can
easily find it.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-18 0:04 ` Stefano Stabellini
@ 2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 21:30 ` Stefano Stabellini
2025-06-23 7:15 ` Jan Beulich
0 siblings, 2 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-19 16:15 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
On 18/06/2025 03:04, Stefano Stabellini wrote:
> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>> Hi Stefano,
>>
>> I'm very sorry for a long silence. Please see my answers below:
>>
>> On 22/05/2025 03:25, Stefano Stabellini wrote:
>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>
>>>> Add chained handling of assigned DT devices to support access-controller
>>>> functionality through SCI framework, so DT device assign request can be
>>>> passed to FW for processing and enabling VM access to requested device
>>>> (for example, device power management through FW interface like SCMI).
>>>>
>>>> The SCI access-controller DT device processing is chained after IOMMU
>>>> processing and expected to be executed for any DT device regardless of its
>>>> protection by IOMMU (or if IOMMU is disabled).
>>>>
>>>> This allows to pass not only IOMMU protected DT device through
>>>> xl.cfg:"dtdev" property for processing:
>>>>
>>>> dtdev = [
>>>> "/soc/video@e6ef0000", <- IOMMU protected device
>>>> "/soc/i2c@e6508000", <- not IOMMU protected device
>>>> ]
>>>>
>>>> The change is done in two parts:
>>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>>>> not fail if DT device is not protected by IOMMU
>>>> 2) add chained call to sci_do_domctl() in do_domctl()
>>>>
>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>> ---
>>>>
>>>>
>>>>
>>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>>>> xen/common/domctl.c | 19 +++++++++++++
>>>> xen/drivers/passthrough/device_tree.c | 6 ++++
>>>> 4 files changed, 76 insertions(+)
>>>>
>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>> index e1522e10e2..8efd541c4f 100644
>>>> --- a/xen/arch/arm/firmware/sci.c
>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>> return 0;
>>>> }
>>>>
>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>> +{
>>>> + struct dt_device_node *dev;
>>>> + int ret = 0;
>>>> +
>>>> + switch ( domctl->cmd )
>>>> + {
>>>> + case XEN_DOMCTL_assign_device:
>>>> + ret = -EOPNOTSUPP;
>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>> The -EOPNOTSUPP code is used because this is part of a chained call after
>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>> XEN_DOMCTL_assign_device
>> call is expected to handle any DT device, regardless of whether the DT
>> device is
>> protected by an IOMMU or if the IOMMU is disabled.
>> The following cases are considered:
>>
>> 1. IOMMU Protected Device (Success)
>>
>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>> we continue
>> processing the DT device by calling sci_do_domctl.
>>
>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>
>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>> disabled,
>> we still proceed to call sci_do_domctl.
> OK this makes sense. I think it is OK to have a special error code to
> say "the IOMMU is disabled" but I don't know if it is a good idea to try
> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
> configuration with domctl disabled, for instance.
>
> It might be wiser to use a different error code. Maybe ENOENT?
>
I see that in the following commit:
71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
-ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
It's not clear to me why this was done from the commit description.
Maybe we should add commit author?
>> 3. Error from iommu_do_domctl (Fail State)
>>
>> If iommu_do_domctl returns any error, the system enters a fail state, and
>> sci_do_domctl is not called.
>>
>> 4. -EOPNOTSUPP from sci_do_domctl
>>
>> If sci_do_domctl returns -EOPNOTSUPP, this indicates one of the following:
>> - The provided device is not a DT device.
>> - There is no cur_mediator available (indicating that the SCI subsystem
>> is enabled
>> in the configuration, but no mediator was provided).
>> - The current mediator does not support assign_dt_device (this is
>> expected to be changed;
>> see below for details).
>> In this case, -EOPNOTSUPP is returned but will be ignored, and the
>> original return value from iommu_do_domctl will be used as the final result.
> Same comment as before. We need to be careful not confuse this case you
> described with other cases where sci_do_domctl is simply not
> implemented.
>
I was trying to mimic iommu_do_domctl logic...
>> 5. Return Code from sci_do_domctl
>>
>> If sci_do_domctl returns 0 (success) or an error code (failure),
>> the return value from iommu_do_domctl is overridden, and the result from
>> sci_do_domctl is returned.
>> Note: -EOPNOTSUPP from iommu_do_domctl will also be overridden since
>> step 2 was successfully completed (or failed).
>>>> + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT )
>>>> + break;
>>> this one
>>>
>>>> + if ( !cur_mediator )
>>>> + break;
>>> this one
>>>
>>>> + if ( !cur_mediator->assign_dt_device )
>>>> + break;
>>> and also this one? It seems more like an -EINVAL as the caller used a
>>> wrong parameter?
>> I think you are right that this case should return -EINVAL because we
>> should fail if mediator
>>
>> without implemented mandatory features was provided. Will be fixed.
>>
>>>> + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
>>>> + domctl->u.assign_device.u.dt.size, &dev);
>>>> + if ( ret )
>>>> + return ret;
>>>> +
>>>> + ret = sci_assign_dt_device(d, dev);
>>>> + if ( ret )
>>>> + break;
>>>> +
>>>> + break;
>>>> + default:
>>>> + /* do not fail here as call is chained with iommu handling */
>>> It looks like this should be an error
>>>
>>>
>>>> + break;
>>>> + }
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> static int __init sci_init(void)
>>>> {
>>>> struct dt_device_node *np;
>>>> diff --git a/xen/arch/arm/include/asm/firmware/sci.h b/xen/arch/arm/include/asm/firmware/sci.h
>>>> index 71fb54852e..b8d1bc8a62 100644
>>>> --- a/xen/arch/arm/include/asm/firmware/sci.h
>>>> +++ b/xen/arch/arm/include/asm/firmware/sci.h
>>>> @@ -146,6 +146,14 @@ int sci_dt_finalize(struct domain *d, void *fdt);
>>>> * control" functionality.
>>>> */
>>>> int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev);
>>>> +
>>>> +/*
>>>> + * SCI domctl handler
>>>> + *
>>>> + * Only XEN_DOMCTL_assign_device is handled for now.
>>>> + */
>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
>>>> #else
>>>>
>>>> static inline bool sci_domain_is_enabled(struct domain *d)
>>>> @@ -195,6 +203,12 @@ static inline int sci_assign_dt_device(struct domain *d,
>>>> return 0;
>>>> }
>>>>
>>>> +static inline int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>> +{
>>>> + return 0;
>>>> +}
>>>> +
>>>> #endif /* CONFIG_ARM_SCI */
>>>>
>>>> #endif /* __ASM_ARM_SCI_H */
>>>> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
>>>> index 05abb581a0..a74ee92067 100644
>>>> --- a/xen/common/domctl.c
>>>> +++ b/xen/common/domctl.c
>>>> @@ -27,6 +27,7 @@
>>>> #include <xen/vm_event.h>
>>>> #include <xen/monitor.h>
>>>> #include <asm/current.h>
>>>> +#include <asm/firmware/sci.h>
>>>> #include <asm/irq.h>
>>>> #include <asm/page.h>
>>>> #include <asm/p2m.h>
>>>> @@ -851,6 +852,24 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>> case XEN_DOMCTL_deassign_device:
>>>> case XEN_DOMCTL_get_device_group:
>>>> ret = iommu_do_domctl(op, d, u_domctl);
>>>> +
>>>> + if ( !ret || ret == -EOPNOTSUPP )
>>> It is better to invert the check:
>>>
>>> if ( ret < 0 && ret != -EOPNOTSUPP )
>>> return ret;
>> +
>>>> + {
>>>> + int ret1;
>>>> + /*
>>>> + * Add chained handling of assigned DT devices to support
>>>> + * access-controller functionality through SCI framework, so
>>>> + * DT device assign request can be passed to FW for processing and
>>>> + * enabling VM access to requested device.
>>>> + * The access-controller DT device processing is chained after IOMMU
>>>> + * processing and expected to be executed for any DT device
>>>> + * regardless if DT device is protected by IOMMU or not (or IOMMU
>>>> + * is disabled).
>>>> + */
>>>> + ret1 = sci_do_domctl(op, d, u_domctl);
>>>> + if ( ret1 != -EOPNOTSUPP )
>>>> + ret = ret1;
>>>> + }
>>>> break;
>>>>
>>>> case XEN_DOMCTL_get_paging_mempool_size:
>>>> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
>>>> index 075fb25a37..2624767e51 100644
>>>> --- a/xen/drivers/passthrough/device_tree.c
>>>> +++ b/xen/drivers/passthrough/device_tree.c
>>>> @@ -318,6 +318,12 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
>>>> break;
>>>> }
>>>>
>>>> + if ( !dt_device_is_protected(dev) )
>>>> + {
>>>> + ret = 0;
>>>> + break;
>>>> + }
>>> I am concerned about this: previously we would call
>>> iommu_assign_dt_device and the same check at the beginning of
>>> iommu_assign_dt_device would return -EINVAL. Now it is a success.
>>>
>>> I am not sure this is appropriate. I wonder if instead we should:
>>>
>>> - remove this chunk from the patch
>>> - change the return error for !dt_device_is_protected at the top of
>>> iommu_assign_dt_device from -EINVAL to -EOPNOTSUPP
>>> - this would fall into the same ret != -EOPNOTSUPP check after
>>> iommu_do_domctl
>> That's a good point. I think we should do the same for
>>
>> > if ( !is_iommu_enabled(d) )
>>
>> > return -EINVAL;
>>
>> because in this case we should process sci as well. I will do the change
>>
>>>> ret = iommu_assign_dt_device(d, dev);
>>>>
>>>> if ( ret )
>>>> --
>>>> 2.34.1
>>> >
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-17 23:38 ` Stefano Stabellini
2025-06-18 7:22 ` Julien Grall
@ 2025-06-19 16:15 ` Oleksii Moisieiev
1 sibling, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-19 16:15 UTC (permalink / raw)
To: Stefano Stabellini, Grygorii Strashko
Cc: Bertrand Marquis, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk
On 18/06/2025 02:38, Stefano Stabellini wrote:
> On Thu, 12 Jun 2025, Grygorii Strashko wrote:
>> On 02.06.25 10:17, Bertrand Marquis wrote:
>>> Hi Stefano and Oleksii,
>>>
>>>> On 23 May 2025, at 22:06, Stefano Stabellini<sstabellini@kernel.org>
>>>> wrote:
>>>>
>>>> One question for Bertrand below
>>>>
>>>>
>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>> This patch introduces SCI driver to support for ARM EL3 Trusted
>>>>> Firmware-A
>>>>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
>>>>> below.
>>>>>
>>>>> +-----------------------------------------+
>>>>> | |
>>>>> | EL3 TF-A SCMI |
>>>>> +-------+--+-------+--+-------+--+-------++
>>>>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
>>>>> +-----+-+ +---+---+ +--+----+ +---+---+
>>>>> smc-id0 | | | |
>>>>> agent0 | | | |
>>>>> +-----v--------+---------+-----------+----+
>>>>> | | | | |
>>>>> | | | | |
>>>>> +--------------+---------+-----------+----+
>>>>> smc-id1 | smc-id2| smc-idX|
>>>>> agent1 | agent2 | agentX |
>>>>> | | |
>>>>> +----v---+ +--v-----+ +--v-----+
>>>>> | | | | | |
>>>>> | Dom0 | | Dom1 | | DomX |
>>>>> | | | | | |
>>>>> | | | | | |
>>>>> +--------+ +--------+ +--------+
>>>>>
>>>>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC
>>>>> shared
>>>>> memory transport for every Agent in the system.
>>>>>
>>>>> The SCMI Agent transport channel defined by pair:
>>>>> - smc-id: SMC/HVC id used for Doorbell
>>>>> - shmem: shared memory for messages transfer, Xen page aligned,
>>>>> p2m_mmio_direct_nc.
>>>>>
>>>>> The follwoing SCMI Agents expected to be defined by SCMI FW to enable
>>>>> SCMI
>>>>> multi-agent functionality under Xen:
>>>>> - Xen manegement agent: trusted agents that accesses to the Base
>>>>> Protocol
>>>>> commands to configure agent specific permissions
>>>>> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
>>>>> allowed direct HW access. At least one OSPM VM agent has to be
>>>>> provided
>>>>> by FW if HW is handled only by Dom0 or Driver Domain.
>>>>>
>>>>> The EL3 SCMI FW expected to implement following Base protocol messages:
>>>>> - BASE_DISCOVER_AGENT
>>>>> - BASE_RESET_AGENT_CONFIGURATION (optional)
>>>>> - BASE_SET_DEVICE_PERMISSIONS (optional)
>>>>>
>>>>> The SCI SCMI SMC multi-agent driver implements following functionality:
>>>>> - It's initialized based on the Host DT SCMI node (only one SCMI
>>>>> interface
>>>>> is supported) which describes Xen management agent SCMI interface.
>>>>>
>>>>> scmi_shm_0 : sram@47ff0000 {
>>>>> compatible = "arm,scmi-shmem";
>>>>> reg = <0x0 0x47ff0000 0x0 0x1000>;
>>>>> };
>>>>> firmware {
>>>>> scmi: scmi {
>>>>> compatible = "arm,scmi-smc";
>>>>> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
>>>> some extra spaces, it might be a copy/paste error
>>>>
>>>>
>>>>> \#address-cells = < 1>;
>>>>> \#size-cells = < 0>;
>>>>> \#access-controller - cells = < 1>;
>>>>> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
>>>>>
>>>>> protocol@X{
>>>>> };
>>>>> };
>>>>> };
>>>>>
>>>>> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
>>>>> probes Agents and build SCMI Agents list; The Agents configuration is
>>>>> taken from:
>>>>>
>>>>> chosen {
>>>>> xen,scmi-secondary-agents = <
>>>>> 1 0x82000003 &scmi_shm_1
>>>>> 2 0x82000004 &scmi_shm_2
>>>>> 3 0x82000005 &scmi_shm_3
>>>>> 4 0x82000006 &scmi_shm_4>;
>>>>> }
>>>>>
>>>>> /{
>>>>> scmi_shm_1: sram@47ff1000 {
>>>>> compatible = "arm,scmi-shmem";
>>>>> reg = <0x0 0x47ff1000 0x0 0x1000>;
>>>>> };
>>>>> scmi_shm_2: sram@47ff2000 {
>>>>> compatible = "arm,scmi-shmem";
>>>>> reg = <0x0 0x47ff2000 0x0 0x1000>;
>>>>> };
>>>>> scmi_shm_3: sram@47ff3000 {
>>>>> compatible = "arm,scmi-shmem";
>>>>> reg = <0x0 0x47ff3000 0x0 0x1000>;
>>>>> };
>>>>> }
>>>>> where first item is "agent_id", second - "arm,smc-id", and third -
>>>>> "arm,scmi-shmem" for
>>>>> this agent_id.
>>>>>
>>>>> Note that Xen is the only one entry in the system which need to know
>>>>> about SCMI multi-agent support.
>>>>>
>>>>> - It implements the SCI subsystem interface required for configuring and
>>>>> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
>>>>> SCMI functionality for domain it has to be configured with unique
>>>>> supported
>>>>> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
>>>>> [smc-id, shmem] defined for this SCMI Agent_id.
>>>>> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
>>>>> -- zero-copy, the guest domain puts SCMI message in shmem;
>>>>> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
>>>>> -- the Xen driver catches exception, do checks and synchronously
>>>>> forwards
>>>>> it to EL3 FW.
>>>>> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
>>>>> management agent channel on domain destroy event. This allows to reset
>>>>> resources used by domain and so implement use-case like domain reboot.
>>>>>
>>>>> Dom0 Enable SCMI SMC:
>>>>> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not
>>>>> provided
>>>>> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0
>>>>> DT.
>>>>> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up
>>>>> shmem
>>>>> node according to assigned agent_id.
>>>>>
>>>>> Guest domains enable SCMI SMC:
>>>>> - xl.cfg: add configuration option as below
>>>>>
>>>>> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
>>>>>
>>>>> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond
>>>>> assigned agent_id for
>>>>> the domain, for example:
>>>>>
>>>>> iomem = [
>>>>> "47ff2,1@22001",
>>>>> ]
>>>> Looking at the code and the configuration options, it looks like it is
>>>> possible to map a scmi-shmem channel at a different address for the
>>>> guest. It seems like it would work. Is that correct?
>>>>
>>>>
>>>>> - DT: add SCMI nodes to the Driver domain partial device tree as in the
>>>>> below example. The "arm,smc-id" should correspond assigned agent_id for
>>>>> the domain:
>>>>>
>>>>> passthrough {
>>>>> scmi_shm_0: sram@22001000 {
>>>>> compatible = "arm,scmi-shmem";
>>>>> reg = <0x0 0x22001000 0x0 0x1000>;
>>>>> };
>>>>>
>>>>> firmware {
>>>>> compatible = "simple-bus";
>>>>> scmi: scmi {
>>>>> compatible = "arm,scmi-smc";
>>>>> arm,smc-id = <0x82000004>;
>>>>> shmem = <&scmi_shm_0>;
>>>>> ...
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>> SCMI "4.2.1.1 Device specific access control"
>>>>>
>>>>> The XEN SCI SCMI SMC multi-agent driver performs "access-controller"
>>>>> provider function
>>>>> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access
>>>>> control" and provides the
>>>>> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an
>>>>> agents have access to.
>>>>> The DT SCMI node should "#access-controller-cells=<1>" property and DT
>>>>> devices should be bound
>>>>> to the Xen SCMI.
>>>>>
>>>>> &i2c1 {
>>>>> access-controllers = <&scmi 0>;
>>>>> };
>>>>>
>>>>> The Dom0 and dom0less domains DT devices will be processed automatically
>>>>> through
>>>>> sci_assign_dt_device() call, but to assign SCMI devices from toolstack
>>>>> the xl.cfg:"dtdev" property
>>>>> shell be used:
>>>>>
>>>>> dtdev = [
>>>>> "/soc/i2c@e6508000",
>>>>> ]
>>>>>
>>>>> xl.cfg:dtdev will contain all nodes which are under SCMI management (not
>>>>> only those which are behind IOMMU).
>>>>>
>>>>> [1]
>>>>> https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
>>>>> [2]
>>>>> https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
>>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>> Thanks for the long explanation, great work! I am really looking forward
>>>> to have this feature in the tree soon.
>>>>
>>>>
>>>>> ---
>>>>>
>>>>> Changes in v4:
>>>>> - toolstack comments from Anthony PERARD
>>>>> - added dom0less support
>>>>> - added doc for "xen,scmi-secondary-agents"
>>>>>
>>>>> docs/man/xl.cfg.5.pod.in | 13 +
>>>>> docs/misc/arm/device-tree/booting.txt | 60 ++
>>>>> docs/misc/xen-command-line.pandoc | 9 +
>>>>> tools/libs/light/libxl_arm.c | 4 +
>>>>> tools/libs/light/libxl_types.idl | 4 +-
>>>>> tools/xl/xl_parse.c | 12 +
>>>>> xen/arch/arm/dom0less-build.c | 11 +
>>>>> xen/arch/arm/domain_build.c | 3 +-
>>>>> xen/arch/arm/firmware/Kconfig | 11 +
>>>>> xen/arch/arm/firmware/Makefile | 1 +
>>>>> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
>>>>> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
>>>>> xen/arch/arm/firmware/scmi-shmem.h | 45 +
>>>>> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
>>>>> xen/include/public/arch-arm.h | 3 +
>>>>> 15 files changed, 1371 insertions(+), 2 deletions(-)
>>>>> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
>>>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
>>>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
>>>>> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
>>>>>
>>>>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>>>>> index 1ccf50b8ea..302c46d8bc 100644
>>>>> --- a/docs/man/xl.cfg.5.pod.in
>>>>> +++ b/docs/man/xl.cfg.5.pod.in
>>>>> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
>>>>> Should be used together with B<dom0_scmi_smc_passthrough> Xen command
>>>>> line
>>>>> option.
>>>>>
>>>>> +=item B<scmi_smc_multiagent>
>>>>> +
>>>>> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI
>>>>> over
>>>>> +SMC calls forwarding from domain to the EL3 firmware (like Trusted
>>>>> Firmware-A)
>>>>> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
>>>>> +specified for the guest.
>>>>> +
>>>>> =back
>>>>>
>>>>> +=item B<agent_id=NUMBER>
>>>>> +
>>>>> +Specifies a non-zero ARM SCI agent id for the guest. This option is
>>>>> mandatory
>>>>> +if the SCMI SMC support is enabled for the guest. The agent ids of
>>>>> domains
>>>>> +existing on a single host must be unique and in the range [1..255].
>>>>> +
>>>>> =back
>>>>>
>>>>> =back
>>>>> diff --git a/docs/misc/arm/device-tree/booting.txt
>>>>> b/docs/misc/arm/device-tree/booting.txt
>>>>> index 8943c04173..c8923ab8b2 100644
>>>>> --- a/docs/misc/arm/device-tree/booting.txt
>>>>> +++ b/docs/misc/arm/device-tree/booting.txt
>>>>> @@ -296,6 +296,20 @@ with the following properties:
>>>>> Should be used together with dom0_scmi_smc_passthrough Xen command
>>>>> line
>>>>> option.
>>>>>
>>>>> + - "scmi_smc_multiagent"
>>>>> +
>>>>> + Enables ARM SCMI SMC multi-agent support for the guest by enabling
>>>>> SCMI over
>>>>> + SMC calls forwarding from domain to the EL3 firmware (like ARM
>>>>> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
>>>>> + The SCMI agent_id should be specified for the guest with
>>>>> "xen,sci_agent_id"
>>>>> + property.
>>>>> +
>>>>> +- "xen,sci_agent_id"
>>>>> +
>>>>> + Specifies a non-zero ARM SCI agent id for the guest. This option is
>>>>> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled
>>>>> for
>>>>> + the guest. The agent ids of guest must be unique and in the range
>>>>> [1..255].
>>>>> +
>>>>> Under the "xen,domain" compatible node, one or more sub-nodes are present for the DomU
>>>>> kernel and ramdisk. @@ -764,3 +778,49 @@ The automatically
>>>>> allocated static shared memory will get mapped at 0x80000000 in
>>>>> DomU1 guest physical address space, and at 0x90000000 in DomU2
>>>>> guest physical address space. DomU1 is explicitly defined as the
>>>>> owner domain, and DomU2 is the borrower domain. + +SCMI SMC
>>>>> multi-agent support +============================ + +For enabling
>>>>> the ARM SCMI SMC multi-agent support (enabled by
>>>>> CONFIG_SCMI_SMC_MA) +the Xen specific SCMI Agent's configuration
>>>>> shell be provided in the Host DT +according to the SCMI compliant
>>>>> EL3 Firmware specification with +ARM SMC/HVC transport using
>>>>> property "xen,scmi-secondary-agents" under
>>>>> +the top-level "chosen" node:
>>>>> +
>>>>> +- xen,scmi-secondary-agents
>>>>> +
>>>>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW
>>>>> and
>>>>> + available for Xen. Each Agent defined as triple consisting of:
>>>>> + SCMI agent_id,
>>>>> + SMC/HVC function_id assigned for the agent transport
>>>>> ("arm,smc-id"),
>>>>> + phandle to SCMI SHM assigned for the agent transport
>>>>> ("arm,scmi-shmem").
>>>>> +
>>>>> +As an example:
>>>>> +
>>>>> +chosen {
>>>>> + xen,scmi-secondary-agents = <
>>>>> + 1 0x82000003 &scmi_shm_1
>>>>> + 2 0x82000004 &scmi_shm_2
>>>>> + 3 0x82000005 &scmi_shm_3
>>>>> + 4 0x82000006 &scmi_shm_4>;
>>>>> +}
>>>> NIT: it should be };
>>>>
>>>> Looking at scmi_probe, collect_agents, and the following SCMI
>>>> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
>>>> information?
>>>>
>>>> It looks like we can discover the agend_ids for every channel, I guess
>>>> what we need to know is the shmem location for every channel? But the
>>>> full list of shmem channel is available below thanks to the scmi-shmem
>>>> nodes.
>>>>
>>>> So, we have the list of scmi-shmem anyway, and we can probe the
>>>> agent_id. The only parameter left is the smc_id/func_id.
>>>>
>>>> Or maybe smc_id/func_id can be calculated from agent_id?
>>>>
>>>> I am asking mostly because if a user is supposed to add this
>>>> xen,scmi-secondary-agents property, where are they supposed to find the
>>>> smc_id/func_id information?
>>>>
>>>> It is important that we write down in this document how the user is
>>>> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
>>>>
>>>>
>>>>> +/{
>>>>> + scmi_shm_1: sram@47ff1000 {
>>>>> + compatible = "arm,scmi-shmem";
>>>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>>>>> + };
>>>>> + scmi_shm_2: sram@47ff2000 {
>>>>> + compatible = "arm,scmi-shmem";
>>>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>>>>> + };
>>>>> + scmi_shm_3: sram@47ff3000 {
>>>>> + compatible = "arm,scmi-shmem";
>>>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>>>>> + };
>>>>> + scmi_shm_3: sram@47ff4000 {
>>>>> + compatible = "arm,scmi-shmem";
>>>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>>>>> + };
>>>> Are these scmi_shm_1 - scmi_shm_3 under the top level device tree node?
>>>> Or are under /firmware? Or are they under /chosen?
>>>>
>>>> I take they are under the top level node together with scmi_shm_0?
>>>>
>>>> Can you please also clarify in the document as well?
>>>>
>>>>
>>>>> +}
>>>>> diff --git a/docs/misc/xen-command-line.pandoc
>>>>> b/docs/misc/xen-command-line.pandoc
>>>>> index 8e50f6b7c7..bc3c64d6ec 100644
>>>>> --- a/docs/misc/xen-command-line.pandoc
>>>>> +++ b/docs/misc/xen-command-line.pandoc
>>>>> @@ -1091,6 +1091,15 @@ which serves as Driver domain. The SCMI will be
>>>>> disabled for Dom0/hwdom and
>>>>> SCMI nodes removed from Dom0/hwdom device tree.
>>>>> (for example, thin Dom0 with Driver domain use-case).
>>>>>
>>>>> +### dom0_scmi_agent_id (ARM)
>>>>> +> `= <integer>`
>>>>> +
>>>>> +The option is available when `CONFIG_SCMI_SMC_MA` is compiled in, and
>>>>> allows to
>>>>> +enable SCMI functionality for Dom0 by specifying a non-zero ARM SCMI
>>>>> agent id.
>>>>> +The SCMI will be disabled for Dom0 if this option is not specified
>>>>> +(for example, thin Dom0 or dom0less use-cases).
>>>>> +The agent ids of domains existing on a single host must be unique.
>>>>> +
>>>>> ### dtuart (ARM)
>>>>>> `= path [:options]`
>>>>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>>>>> index 28ba9eb787..7712f53cd4 100644
>>>>> --- a/tools/libs/light/libxl_arm.c
>>>>> +++ b/tools/libs/light/libxl_arm.c
>>>>> @@ -229,6 +229,10 @@ int libxl__arch_domain_prepare_config(libxl__gc
>>>>> *gc,
>>>>> case LIBXL_ARM_SCI_TYPE_SCMI_SMC:
>>>>> config->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>>>> break;
>>>>> + case LIBXL_ARM_SCI_TYPE_SCMI_SMC_MULTIAGENT:
>>>>> + config->arch.arm_sci_type =
>>>>> XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>>>> + config->arch.arm_sci_agent_id =
>>>>> d_config->b_info.arch_arm.arm_sci.agent_id;
>>>>> + break;
>>>>> default:
>>>>> LOG(ERROR, "Unknown ARM_SCI type %d",
>>>>> d_config->b_info.arch_arm.arm_sci.type);
>>>>> diff --git a/tools/libs/light/libxl_types.idl
>>>>> b/tools/libs/light/libxl_types.idl
>>>>> index aa2190ab5b..11e31ce786 100644
>>>>> --- a/tools/libs/light/libxl_types.idl
>>>>> +++ b/tools/libs/light/libxl_types.idl
>>>>> @@ -553,11 +553,13 @@ libxl_sve_type = Enumeration("sve_type", [
>>>>>
>>>>> libxl_arm_sci_type = Enumeration("arm_sci_type", [
>>>>> (0, "none"),
>>>>> - (1, "scmi_smc")
>>>>> + (1, "scmi_smc"),
>>>>> + (2, "scmi_smc_multiagent")
>>>>> ], init_val = "LIBXL_ARM_SCI_TYPE_NONE")
>>>>>
>>>>> libxl_arm_sci = Struct("arm_sci", [
>>>>> ("type", libxl_arm_sci_type),
>>>>> + ("agent_id", uint8)
>>>>> ])
>>>>>
>>>>> libxl_rdm_reserve = Struct("rdm_reserve", [
>>>>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>>>>> index bd22be9d33..81aa3797e3 100644
>>>>> --- a/tools/xl/xl_parse.c
>>>>> +++ b/tools/xl/xl_parse.c
>>>>> @@ -1306,6 +1306,18 @@ static int parse_arm_sci_config(XLU_Config *cfg,
>>>>> libxl_arm_sci *arm_sci,
>>>>> }
>>>>> }
>>>>>
>>>>> + if (MATCH_OPTION("agent_id", ptr, oparg)) {
>>>>> + unsigned long val = parse_ulong(oparg);
>>>>> +
>>>>> + if (!val || val > 255) {
>>>>> + fprintf(stderr, "An invalid ARM_SCI agent_id specified
>>>>> (%lu). Valid range [1..255]\n",
>>>>> + val);
>>>>> + ret = ERROR_INVAL;
>>>>> + goto parse_error;
>>>>> + }
>>>>> + arm_sci->agent_id = val;
>>>>> + }
>>>>> +
>>>>> ptr = strtok(NULL, ",");
>>>>> }
>>>>>
>>>>> diff --git a/xen/arch/arm/dom0less-build.c
>>>>> b/xen/arch/arm/dom0less-build.c
>>>>> index 0a00f03a25..43d21eb889 100644
>>>>> --- a/xen/arch/arm/dom0less-build.c
>>>>> +++ b/xen/arch/arm/dom0less-build.c
>>>>> @@ -835,6 +835,17 @@ int __init domu_dt_sci_parse(struct dt_device_node
>>>>> *node,
>>>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_NONE;
>>>>> else if ( !strcmp(sci_type, "scmi_smc") )
>>>>> d_cfg->arch.arm_sci_type = XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC;
>>>>> + else if ( !strcmp(sci_type, "scmi_smc_multiagent") )
>>>>> + {
>>>>> + uint32_t agent_id = 0;
>>>>> +
>>>>> + if ( !dt_property_read_u32(node, "xen,sci_agent_id", &agent_id)
>>>>> ||
>>>>> + !agent_id )
>>>> shouldn't we check that agent_id <= 255 ?
>>>>
>>>>
>>>>> + return -EINVAL;
>>>>> +
>>>>> + d_cfg->arch.arm_sci_type =
>>>>> XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC_MA;
>>>>> + d_cfg->arch.arm_sci_agent_id = agent_id;
>>>>> + }
>>>>> else
>>>>> {
>>>>> printk(XENLOG_ERR "xen,sci_type in not valid (%s) for domain
>>>>> %s\n",
>>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>>> index 36d28b52a4..0c9274a2b3 100644
>>>>> --- a/xen/arch/arm/domain_build.c
>>>>> +++ b/xen/arch/arm/domain_build.c
>>>>> @@ -616,7 +616,8 @@ static int __init write_properties(struct domain *d,
>>>>> struct kernel_info *kinfo,
>>>>> dt_property_name_is_equal(prop,
>>>>> "linux,uefi-mmap-start") ||
>>>>> dt_property_name_is_equal(prop,
>>>>> "linux,uefi-mmap-size") ||
>>>>> dt_property_name_is_equal(prop,
>>>>> "linux,uefi-mmap-desc-size") ||
>>>>> - dt_property_name_is_equal(prop,
>>>>> "linux,uefi-mmap-desc-ver"))
>>>>> + dt_property_name_is_equal(prop,
>>>>> "linux,uefi-mmap-desc-ver") ||
>>>>> + dt_property_name_is_equal(prop,
>>>>> "xen,scmi-secondary-agents") )
>>>>> continue;
>>>>>
>>>>> if ( dt_property_name_is_equal(prop, "xen,dom0-bootargs") ) diff --git a/xen/arch/arm/firmware/Kconfig
>>>>> b/xen/arch/arm/firmware/Kconfig index 5c5f0880c4..6b051c8ada
>>>>> 100644 --- a/xen/arch/arm/firmware/Kconfig +++
>>>>> b/xen/arch/arm/firmware/Kconfig @@ -29,6 +29,17 @@ config SCMI_SMC
>>>>> driver domain. Use with EL3 firmware which supports only single
>>>>> SCMI OSPM agent. +config SCMI_SMC_MA + bool "Enable ARM SCMI SMC multi-agent driver"
>>>>> + select ARM_SCI
>>>>> + help
>>>>> + Enables SCMI SMC/HVC multi-agent in XEN to pass SCMI requests from
>>>>> Domains
>>>>> + to EL3 firmware (TF-A) which supports multi-agent feature.
>>>>> + This feature allows to enable SCMI per Domain using unique SCMI
>>>>> agent_id,
>>>>> + so Domain is identified by EL3 firmware as an SCMI Agent and can
>>>>> access
>>>>> + allowed platform resources through dedicated SMC/HVC Shared memory
>>>>> based
>>>>> + transport.
>>>>> +
>>>>> endchoice
>>>>>
>>>>> endmenu
>>>>> diff --git a/xen/arch/arm/firmware/Makefile
>>>>> b/xen/arch/arm/firmware/Makefile
>>>>> index 71bdefc24a..37927e690e 100644
>>>>> --- a/xen/arch/arm/firmware/Makefile
>>>>> +++ b/xen/arch/arm/firmware/Makefile
>>>>> @@ -1,2 +1,3 @@
>>>>> obj-$(CONFIG_ARM_SCI) += sci.o
>>>>> obj-$(CONFIG_SCMI_SMC) += scmi-smc.o
>>>>> +obj-$(CONFIG_SCMI_SMC_MA) += scmi-shmem.o scmi-smc-multiagent.o
>>>>> diff --git a/xen/arch/arm/firmware/scmi-proto.h
>>>>> b/xen/arch/arm/firmware/scmi-proto.h
>>>>> new file mode 100644
>>>>> index 0000000000..3f4b9c5d6b
>>>>> --- /dev/null
>>>>> +++ b/xen/arch/arm/firmware/scmi-proto.h
>>>>> @@ -0,0 +1,164 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>>> +/*
>>>>> + * Arm System Control and Management Interface definitions
>>>>> + * Version 3.0 (DEN0056C)
>>>>> + *
>>>>> + * Copyright (c) 2024 EPAM Systems
>>>>> + */
>>>>> +
>>>>> +#ifndef XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>>>>> +#define XEN_ARCH_ARM_SCI_SCMI_PROTO_H_
>>>> NIT: ARM_FIRMWARE_SCMI_PROTO_H
>>>>
>>>>
>>>>> +#include <xen/stdint.h>
>>>>> +
>>>>> +#define SCMI_SHORT_NAME_MAX_SIZE 16
>>>>> +
>>>>> +/* SCMI status codes. See section 4.1.4 */
>>>>> +#define SCMI_SUCCESS 0
>>>>> +#define SCMI_NOT_SUPPORTED (-1)
>>>>> +#define SCMI_INVALID_PARAMETERS (-2)
>>>>> +#define SCMI_DENIED (-3)
>>>>> +#define SCMI_NOT_FOUND (-4)
>>>>> +#define SCMI_OUT_OF_RANGE (-5)
>>>>> +#define SCMI_BUSY (-6)
>>>>> +#define SCMI_COMMS_ERROR (-7)
>>>>> +#define SCMI_GENERIC_ERROR (-8)
>>>>> +#define SCMI_HARDWARE_ERROR (-9)
>>>>> +#define SCMI_PROTOCOL_ERROR (-10)
>>>>> +
>>>>> +/* Protocol IDs */
>>>>> +#define SCMI_BASE_PROTOCOL 0x10
>>>>> +
>>>>> +/* Base protocol message IDs */
>>>>> +#define SCMI_BASE_PROTOCOL_VERSION 0x0
>>>>> +#define SCMI_BASE_PROTOCOL_ATTIBUTES 0x1
>>>>> +#define SCMI_BASE_PROTOCOL_MESSAGE_ATTRIBUTES 0x2
>>>>> +#define SCMI_BASE_DISCOVER_AGENT 0x7
>>>>> +#define SCMI_BASE_SET_DEVICE_PERMISSIONS 0x9
>>>>> +#define SCMI_BASE_RESET_AGENT_CONFIGURATION 0xB
>>>>> +
>>>>> +typedef struct scmi_msg_header {
>>>>> + uint8_t id;
>>>>> + uint8_t type;
>>>>> + uint8_t protocol;
>>>>> + uint32_t status;
>>>>> +} scmi_msg_header_t;
>>>>> +
>>>>> +/* Table 2 Message header format */
>>>>> +#define SCMI_HDR_ID GENMASK(7, 0)
>>>>> +#define SCMI_HDR_TYPE GENMASK(9, 8)
>>>>> +#define SCMI_HDR_PROTO GENMASK(17, 10)
>>>>> +
>>>>> +#define SCMI_FIELD_GET(_mask, _reg)
>>>>> \
>>>>> + ((typeof(_mask))(((_reg) & (_mask)) >> (ffs64(_mask) - 1)))
>>>>> +#define SCMI_FIELD_PREP(_mask, _val)
>>>>> \
>>>>> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
>>>>> +
>>>>> +static inline uint32_t pack_scmi_header(scmi_msg_header_t *hdr)
>>>>> +{
>>>>> + return SCMI_FIELD_PREP(SCMI_HDR_ID, hdr->id) |
>>>>> + SCMI_FIELD_PREP(SCMI_HDR_TYPE, hdr->type) |
>>>>> + SCMI_FIELD_PREP(SCMI_HDR_PROTO, hdr->protocol);
>>>>> +}
>>>>> +
>>>>> +static inline void unpack_scmi_header(uint32_t msg_hdr,
>>>>> scmi_msg_header_t *hdr)
>>>>> +{
>>>>> + hdr->id = SCMI_FIELD_GET(SCMI_HDR_ID, msg_hdr);
>>>>> + hdr->type = SCMI_FIELD_GET(SCMI_HDR_TYPE, msg_hdr);
>>>>> + hdr->protocol = SCMI_FIELD_GET(SCMI_HDR_PROTO, msg_hdr);
>>>>> +}
>>>>> +
>>>>> +static inline int scmi_to_xen_errno(int scmi_status)
>>>>> +{
>>>>> + if ( scmi_status == SCMI_SUCCESS )
>>>>> + return 0;
>>>>> +
>>>>> + switch ( scmi_status )
>>>>> + {
>>>>> + case SCMI_NOT_SUPPORTED:
>>>>> + return -EOPNOTSUPP;
>>>>> + case SCMI_INVALID_PARAMETERS:
>>>>> + return -EINVAL;
>>>>> + case SCMI_DENIED:
>>>>> + return -EACCES;
>>>>> + case SCMI_NOT_FOUND:
>>>>> + return -ENOENT;
>>>>> + case SCMI_OUT_OF_RANGE:
>>>>> + return -ERANGE;
>>>>> + case SCMI_BUSY:
>>>>> + return -EBUSY;
>>>>> + case SCMI_COMMS_ERROR:
>>>>> + return -ENOTCONN;
>>>>> + case SCMI_GENERIC_ERROR:
>>>>> + return -EIO;
>>>>> + case SCMI_HARDWARE_ERROR:
>>>>> + return -ENXIO;
>>>>> + case SCMI_PROTOCOL_ERROR:
>>>>> + return -EBADMSG;
>>>>> + default:
>>>>> + return -EINVAL;
>>>>> + }
>>>>> +}
>>>>> +
>>>>> +/* PROTOCOL_VERSION */
>>>>> +#define SCMI_VERSION_MINOR GENMASK(15, 0)
>>>>> +#define SCMI_VERSION_MAJOR GENMASK(31, 16)
>>>>> +
>>>>> +struct scmi_msg_prot_version_p2a {
>>>>> + uint32_t version;
>>>>> +} __packed;
>>>>> +
>>>>> +/* BASE PROTOCOL_ATTRIBUTES */
>>>>> +#define SCMI_BASE_ATTR_NUM_PROTO GENMASK(7, 0)
>>>>> +#define SCMI_BASE_ATTR_NUM_AGENT GENMASK(15, 8)
>>>>> +
>>>>> +struct scmi_msg_base_attributes_p2a {
>>>>> + uint32_t attributes;
>>>>> +} __packed;
>>>>> +
>>>>> +/*
>>>>> + * BASE_DISCOVER_AGENT
>>>>> + */
>>>>> +#define SCMI_BASE_AGENT_ID_OWN 0xFFFFFFFF
>>>>> +
>>>>> +struct scmi_msg_base_discover_agent_a2p {
>>>>> + uint32_t agent_id;
>>>>> +} __packed;
>>>>> +
>>>>> +struct scmi_msg_base_discover_agent_p2a {
>>>>> + uint32_t agent_id;
>>>>> + char name[SCMI_SHORT_NAME_MAX_SIZE];
>>>>> +} __packed;
>>>>> +
>>>>> +/*
>>>>> + * BASE_SET_DEVICE_PERMISSIONS
>>>>> + */
>>>>> +#define SCMI_BASE_DEVICE_ACCESS_ALLOW BIT(0, UL)
>>>>> +
>>>>> +struct scmi_msg_base_set_device_permissions_a2p {
>>>>> + uint32_t agent_id;
>>>>> + uint32_t device_id;
>>>>> + uint32_t flags;
>>>>> +} __packed;
>>>>> +
>>>>> +/*
>>>>> + * BASE_RESET_AGENT_CONFIGURATION
>>>>> + */
>>>>> +#define SCMI_BASE_AGENT_PERMISSIONS_RESET BIT(0, UL)
>>>>> +
>>>>> +struct scmi_msg_base_reset_agent_cfg_a2p {
>>>>> + uint32_t agent_id;
>>>>> + uint32_t flags;
>>>>> +} __packed;
>>>>> +
>>>>> +#endif /* XEN_ARCH_ARM_SCI_SCMI_PROTO_H_ */
>>>>> +
>>>>> +/*
>>>>> + * Local variables:
>>>>> + * mode: C
>>>>> + * c-file-style: "BSD"
>>>>> + * c-basic-offset: 4
>>>>> + * tab-width: 4
>>>>> + * indent-tabs-mode: nil
>>>>> + * End:
>>>>> + */
>>>>> diff --git a/xen/arch/arm/firmware/scmi-shmem.c
>>>>> b/xen/arch/arm/firmware/scmi-shmem.c
>>>>> new file mode 100644
>>>>> index 0000000000..dd613ee0b5
>>>>> --- /dev/null
>>>>> +++ b/xen/arch/arm/firmware/scmi-shmem.c
>>>>> @@ -0,0 +1,173 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>>> +/*
>>>>> + * SCI SCMI multi-agent driver, using SMC/HVC shmem as transport.
>>>>> + *
>>>>> + * Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>>> + * Copyright (c) 2025 EPAM Systems
>>>>> + */
>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>>> +
>>>>> +#include <asm/io.h>
>>>>> +#include <xen/err.h>
>>>>> +
>>>>> +#include "scmi-proto.h"
>>>>> +#include "scmi-shmem.h"
>>>> This code is written more generically than the description implies. If
>>>> we only want to make SMC calls to TF-A on EL3 and exchange data with it
>>>> over shared memory, then I think:
>>>> - we don't need the __iomem tag, as there is no MMIO
>>> agree
>>>
>>>> - we only need a DMB, not a DSB (readl and writel imply DSB, use only
>>>> readl_relaxed and writel_relaxed)
>>> agree
>>>
>>>> On the other hand, if we also want to handle the case where the SCMI
>>>> server could be on a separate co-processor, then what this code is doing
>>>> is not sufficient because we also need a dcache flush, in addition to
>>>> the DSB.
>>>>
>>>> Bertrand, can you double-check?
>>> If we want to handle a case where the memory is accessible to a coprocessor
>>> but there is no cache coherency, we need to flush the dcache definitely.
>>>
>>> Seeing the amount of data here, I do agree with Stefano that it would be a
>>> good
>>> idea to make the provision to flush the data cache in all cases. Even if the
>>> data
>>> is accessed by a secure partition or the firmware coherently, flushing in
>>> all cases
>>> would have very limited performance impact here.
>>>
>>> There is the other solution to have some kind of parameter to say if the
>>> accessor
>>> has coherent cache access but I do not think the performance impact here
>>> would
>>> justify such a complexity.
>>>
>> The SCMI shmem expected to be mapped as MT_NON_CACHEABLE in all cases.
> Is that part of the spec?
It's not exactly, for shared memory transport there are the following
notes in DEN0056E spec:
5.1.1 - "The caller must ensure the appropriate ordering of memory
operations so that all updates to the shared
memory must be visible to the callee before ringing the doorbell.
Equally, the callee must ensure that all
shared memory changes are visible to the caller before updating the status."
As was stated before, all known (to us) SCMI client expecting SCMI shmem
to be mapped as NON-CACHABLE.
But the SCMI specification includes mapping information specifically for
FastChannels, sensors, performance,
and power shared memory (which are not currently supported in Xen-SCMI):
DEN0056. 5.3 FastChannels must be mapped as non-cached - we don't use it
in xen
4.7.5 Sensor Values Shared Memory - "The memory must be accessible from
the Non-secure world, and OSPM must map it as non-cached normal memory
or device memory."
4.5.5 Performance domain statistics shared memory region - "the shared
memory must be accessible from the Non-secure world and must be mapped as
non-cached normal memory or device memory."
4.3.4 *Power state statistics shared memory region - *The memory must be
accessible from the Non-secure world, and OSPM must map it as non-cached
normal memory or
device memory
>> The Linux does devm_ioremap() -> ioremap() ->
>> (ARM64) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
>>
>> There is also note in docs:
>> "+- shmem: shared memory for messages transfer, **Xen page aligned** with
>> mapping``p2m_mmio_direct_nc``."
>>
>> In the case of SCP - the SCMI shmem can be actually be in SRAM.
>>
>> So, are you sure cache manipulations are required here?
> No, if the memory is mapped as uncacheable everywhere then the cache
> manipulations are not needed. However, we probably still need a dsb.
>
> I understand now why they decided to use __memcpy_fromio in Linux: it is
> not MMIO but they needed a memcpy followed by DSB, so they decided to
> reuse the existing MMIO functions although the buffer is not MMIO.
>
> In Xen I would prefer we just used memcpy followed by DSB. The opinion
> of other maintainers is very welcome.
It sounds reasonable. I can update it as the following, for example:
```
diff --git a/xen/arch/arm/firmware/scmi-shmem.c
b/xen/arch/arm/firmware/scmi-shmem.c
index dd613ee0b5..252d70609c 100644
--- a/xen/arch/arm/firmware/scmi-shmem.c
+++ b/xen/arch/arm/firmware/scmi-shmem.c
@@ -97,14 +97,16 @@ int shmem_put_message(volatile struct
scmi_shared_mem __iomem *shmem,
if ( ret )
return ret;
- writel_relaxed(0x0, &shmem->channel_status);
+ shmem->channel_status = 0x0;
/* Writing 0x0 right now, but "shmem"_FLAG_INTR_ENABLED can be set */
- writel_relaxed(0x0, &shmem->flags);
- writel_relaxed(sizeof(shmem->msg_header) + len, &shmem->length);
- writel(pack_scmi_header(hdr), &shmem->msg_header);
+ shmem->flags = 0x0;
+ shmem->length = sizeof(shmem->msg_header) + len;
+ shmem->msg_header = pack_scmi_header(hdr);
if ( len > 0 && data )
- __memcpy_toio(shmem->msg_payload, data, len);
+ memcpy(shmem->msg_payload, data, len);
+
+ wmb();
return 0;
}
@@ -157,7 +159,9 @@ int shmem_get_response(const volatile struct
scmi_shared_mem __iomem *shmem,
}
if ( recv_len > 0 )
- __memcpy_fromio(data, shmem->msg_payload + pad, recv_len);
+ memcpy(data, shmem->msg_payload + pad, recv_len);
+
+ rmb();
return 0;
}
```
^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-18 7:22 ` Julien Grall
@ 2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 21:50 ` Stefano Stabellini
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-19 16:15 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini, Grygorii Strashko
Cc: Bertrand Marquis, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk
On 18/06/2025 10:22, Julien Grall wrote:
> Hi,
>
> On 18/06/2025 00:38, Stefano Stabellini wrote:
>> On Thu, 12 Jun 2025, Grygorii Strashko wrote:
>>> On 02.06.25 10:17, Bertrand Marquis wrote:
>>>>> On the other hand, if we also want to handle the case where the SCMI
>>>>> server could be on a separate co-processor, then what this code is
>>>>> doing
>>>>> is not sufficient because we also need a dcache flush, in addition to
>>>>> the DSB.
>>>>>
>>>>> Bertrand, can you double-check?
>>>>
>>>> If we want to handle a case where the memory is accessible to a
>>>> coprocessor
>>>> but there is no cache coherency, we need to flush the dcache
>>>> definitely.
>>>>
>>>> Seeing the amount of data here, I do agree with Stefano that it
>>>> would be a
>>>> good
>>>> idea to make the provision to flush the data cache in all cases.
>>>> Even if the
>>>> data
>>>> is accessed by a secure partition or the firmware coherently,
>>>> flushing in
>>>> all cases
>>>> would have very limited performance impact here.
>>>>
>>>> There is the other solution to have some kind of parameter to say
>>>> if the
>>>> accessor
>>>> has coherent cache access but I do not think the performance impact
>>>> here
>>>> would
>>>> justify such a complexity.
>>>>
>>> The SCMI shmem expected to be mapped as MT_NON_CACHEABLE in all cases.
>
> I can't find MT_NON_CACHEABLE anywhere in Xen or Linux. My
> interpretation is that the memory attribute would be normal memory non
> cacheable. However, this doesn't add up with ...
>
Sorry for the confusion. This define was taken from TF-A and it is the
same as Xen MT_NORMAL_NC.
The main idea was to mention that memory is non_cachable.
>>> The Linux does devm_ioremap() -> ioremap() ->
>>> (ARM64) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
>
> ... this line. This is device nGnRE which is a lot more restrictive
> (for instance it doesn't allow unaligned access).
>
In Xen scmi memory is mapped using ioremap_nocache which is mapped as
MT_DEVICE_nGnRE (same as linux).
In TF-A SCMI shared memory is mapped as MT_DEVICE (which is
nGnRE: MAIR_DEV_nGnRE).
Again, sorry for the confusion.
>>>
>>> There is also note in docs:
>>> "+- shmem: shared memory for messages transfer, **Xen page aligned**
>>> with
>>> mapping``p2m_mmio_direct_nc``."
>>>
>>> In the case of SCP - the SCMI shmem can be actually be in SRAM.
>>>
>>> So, are you sure cache manipulations are required here?
>>
>> No, if the memory is mapped as uncacheable everywhere then the cache
>> manipulations are not needed. However, we probably still need a dsb.
>>
>> I understand now why they decided to use __memcpy_fromio in Linux: it is
>> not MMIO but they needed a memcpy followed by DSB, so they decided to
>> reuse the existing MMIO functions although the buffer is not MMIO.
>
> From my understanding, memcpy_fromio() is not just a mempcy() + dsb.
> It also guarantees the access will be aligned (this is not guarantee
> by our memcpy()).
>
From my understanding Linux using memcpy_fromio() because memcpy
function is highly
optimized in linux and will produce exception,and looking into the
memcpy implementation
in Xen (xen/arch/arm/arm64/lib/memcpy.S) I'm not sure if it can be used
instead of memcpy_fromio. Could you please advise how to proceed?
> Now the question is why does Linux map the region Device nGnRE but we
> are mapping non-cacheable?
>
> Cheers,
>
Again sorry for the confusion, we're mapping as MT_DEVICE_nGnRE in Xen
(please see above).
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-18 0:35 ` Stefano Stabellini
@ 2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 21:57 ` Stefano Stabellini
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-19 16:15 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
On 18/06/2025 03:35, Stefano Stabellini wrote:
> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>> On 23/05/2025 23:19, Stefano Stabellini wrote:
>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>
>>>> Proposal description to add separate SCMI DT node for Xen management agent
>>>> under "chosen" or xen-config node, like Hyperlaunch "xen,config".
>>> I think it is OK to place a larger "xen,config" node under /chosen with
>>> more information for Xen to setup SCMI more easily.
>>>
>>>
>>>> This proposal introduces a new approach to the Xen multi-domain
>>>> configuration, where all Xen-specific configuration has been moved
>>>> under the "/chosen" node. This requires less Dom0 device tree
>>>> manipulation and isolates Xen configuration from domain configuration.
>>>>
>>>> This approach provides the following device tree (DT) parameters:
>>>>
>>>> - "xen,scmi-secondary-agents": A Xen-specific parameter under the
>>>> "/chosen" node, which describes the SCMI agent configuration for
>>>> the domains.
>>>> - the SCMI configuration for Xen (privileged agent) and the shared
>>>> memory configuration for all agents are provided under the "/chosen"
>>>> node and are used strictly by Xen for its initial configuration.
>>>> - the scmi_shm and SCMI configuration for Dom0 are placed in the
>>>> "/firmware/scmi" node so that they can be moved to Dom0 without
>>>> any changes.
>>> Isn't the SCMI configuration present in /firmware/scmi referring to the
>>> privileged agent=0 meant to be used by Xen?
>>>
>>> I certainly see benefits in simplifying the configuration and especially
>>> reducing the amount of changes a user might have to make on the
>>> underlying device tree, but if the user needs to change /firmware/scmi
>>> with the Dom0 information, it seems more dangerous and error prone than
>>> the previous approach.
>>>
>> The idea is to move the privileged agent=0 configuration to the /chosen
>> node and
>>
>> assign agent=1 to the Dom0 node under /firmware/scmi.
>>
>> Benefits of This Approach:
>> - No Modification of the Xen DT Node Required
>>
>> This eliminates the need to modify the Xen Device Tree (DT) node
>> before creating Dom0 in
>>
>> order to set the correct shared memory (shmem).
>>
>> -Consistent SCMI Configuration Format
>>
>> The Dom0 DT will have the same SCMI configuration format as other
>> domains, simplifying the
>>
>> overall configuration process.
>>
>> - Unified SCMI Configuration Method
>>
>> There will no longer be a need to use a different approach for SCMI
>> configuration in Dom0
>>
>> compared to other domains.
>>
>> - Separation Between Dom0 and Privileged Node
>>
>> This provides a clear separation between the Dom0 node and the
>> privileged node.
>>
>> For example:
>> If Dom0 only requires the clock protocol, but the Xen SCMI
>> configuration requires additional protocols,
>>
>> this approach allows Dom0 to receive only the necessary
>> protocol configuration.
> I don't think this is a good idea because we end up confusing the data
> for Xen and the data for the DomUs/Dom0 in the host device tree.
>
> I think we should follow these very simple guidelines:
>
> - The host DTB (the DTB given to Xen at boot) should be the same for Xen
> and for Linux baremetal (no KVM), with the exception of the data under
> the /chosen node
>
> - We can place Xen specific configurations under the /chosen node in the
> host DTB, both Xen hypervisor configuration and also Dom0/DomU
> configurations
>
> This way, the host information remains generic and the configuration for
> Xen the domUs/Dom0 is kept clearly separate from the rest. I don't
> think we can break these two assumptions but we have more freedom with
> the rest.
>
> If we start with these two simple assumptions, here are the
> consequences:
>
> - data under /firmware/scmi should be the same for Xen and baremetal
> Linux, ideally it would describe Xen's agent0 channel in both cases
According to the proposal:
The data under the Host DT /firmware/scmi node will always point to the
default OSPM agent, which will remain
the same (smc-id and shmem) for both the BSP case (no Xen) and the Xen
case (Dom0 domain).
Meanwhile, the Xen management agent's SCMI node and configuration are
expected to be placed under /chosen.
This approach ensures that the Host DT remains as unchanged as possible.
Currently:
The Host DT /firmware/scmi node requires modification to point to the
Xen management agent by changing
the smc-id and shmem values.
At boot time, during Dom0 creation, the SCMI multi-agent driver reverts
these changes.
> - We can add as many nodes as we like under /chosen, including a
> xen,config node and also additional nodes for the domains config
>
> - We can define the new nodes under /chosen to be as simple as possible
> for the user to configure them, while also trying to minimize
> complexity in Xen in terms of DT manipulations
>
>
>
> If the Xen SCMI configuration data cannot be the same as the Linux
> baremetal SCMI configuration (i.e. /firmware/scmi has to be different in
> the two cases) I would still suggest to avoid modifying /firmware/scmi
> for Xen and instead provide the Xen configuration under /chosen. It is
> important to keep everything in the host DTB (except /chosen) the same
> between Linux baremetal and Xen.
>
> However, we can add a new node similar to /firmware/scmi under /chosen
> specifically for Xen, such as /chosen/xen-config/scmi
>
> The Dom0 configuration cannot be expected to be under /firmware/scmi.
> However, it could also be defined under /chosen.
No. The idea is to keep it and provide unchanged, but with possibility
to "disable SCMI for Dom0"
- now with Xen bootarg parameter.
> Keep in mind that the more we add to /chosen the more difficult it will
> be for the user to configure the system. I think we should plan ahead to
> have ImageBuilder be able to generate the DT nodes under /chosen for Xen
> starting from the simplest possible configuration format provided by the
> user. The more complex and rich are the device tree nodes under /chosen,
> the more important is the documentation and ImageBuilder support for it.
>
>
Regarding all the other points you’ve mentioned – this is exactly what
we are trying to achieve
with this proposal.
We are proposing the following changes to the approach so that all
requirements are met, and the
Xen device tree (DT) will remain the same as the Host Platform DT (BSP
Linux), except for the /chosen node:
```
/{
chosen {
...
// Xen SCMI management channel
scmi_shm_xen : sram@47ff1000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff1000 0x0 0x1000>;
};
scmi_xen: scmi {
compatible = "arm,scmi-smc";
arm,smc-id = <0x82000003>; <--- Xen manegement agent smc-id
#address-cells = < 1>;
#size-cells = < 0>;
#access-controller-cells = < 1>;
shmem = <&scmi_shm_xen>; <--- Xen manegement agent shmem
};
// SCMI multi-agent configuration
scmi_shm_2: sram@47ff2000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff2000 0x0 0x1000>;
};
scmi_shm_3: sram@47ff3000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff3000 0x0 0x1000>;
};
scmi_shm_4: sram@47ff4000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff4000 0x0 0x1000>;
};
xen,scmi-secondary-agents = <
0x82000002 &scmi_shm 1
0x82000004 &scmi_shm_2 2
0x82000005 &scmi_shm_3 3
0x82000006 &scmi_shm_4 4>;
};
// Host SCMI OSPM channel - provided to the Dom0 as is if SCMI
enabled for it (same address as Host Platform DT)
scmi_shm: sram@47ff0000 {
compatible = "arm,scmi-shmem";
reg = <0x0 0x47ff0000 0x0 0x1000>;
};
firmware { <-- the below configuration is the same as Host
Platform DT and will be passed to Dom0
scmi: scmi {
compatible = "arm,scmi-smc";
arm,smc-id = <0x82000002>; <--- Host OSPM agent smc-id
which is the same as in Host Platform DT
#address-cells = < 1>;
#size-cells = < 0>;
shmem = <&scmi_shm>; <--- Host OSPM agent shmem (same
address as Host Platform DT)
protocol@X{
};
};
};
}
```
>>>> This configuration allows the use of Xen-specific nodes to provide
>>>> information strictly needed by Xen while using the default SCMI
>>>> configuration for Dom0 and other domains. As a result, no additional
>>>> bindings need to be introduced to the device tree.
>>> This is not actually implemented by this patch series, right?
>> It is not. Just posted this document as a proposal.
>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>> ---
>>>>
>>>>
>>>>
>>>> .../arm/firmware/arm-scmi-proposal.rst | 224 ++++++++++++++++++
>>>> 1 file changed, 224 insertions(+)
>>>> create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
>>>>
>>>> diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
>>>> new file mode 100644
>>>> index 0000000000..fcc2ed2b65
>>>> --- /dev/null
>>>> +++ b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
>>>> @@ -0,0 +1,224 @@
>>>> +
>>>> +Proposal for SCMI multi-agent driver bindings
>>>> +=============================================
>>>> +
>>>> +Now the Xen configuration for SCMI multi-agent support is done in a bit complicated way, especially
>>>> +from SCMI multi-agent driver initialization and Dom0 DT manipulation point of view.
>>>> +Also it does not take into account future requirements to support SCP SCMI FW.
>>>> +
>>>> +To enable SCMI multi-agent user need:
>>>> +
>>>> +* take host DT with basic SCMI enabled
>>>> +* add SCMI shared-memory nodes for all agents
>>>> +* update SCMI node to point on SCMI Xen management channel (``[smc-id, shmem]``)
>>>> +* add "xen,scmi-secondary-agents" property to the "\chosen" node
>>>> +
>>>> +.. code::
>>>> +
>>>> + chosen {
>>>> + xen,scmi-secondary-agents = <
>>>> + 1 0x82000003 &scmi_shm_1
>>>> + 2 0x82000004 &scmi_shm_2
>>>> + 3 0x82000005 &scmi_shm_3
>>>> + 4 0x82000006 &scmi_shm_4>;
>>>> + }
>>>> +
>>>> + /{
>>>> + // SCMI shared-memory nodes for all agents
>>>> + scmi_shm_0 : sram@47ff0000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_1: sram@47ff1000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_2: sram@47ff2000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_3: sram@47ff3000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_4: sram@47ff4000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>>>> + };
>>>> +
>>>> + firmware {
>>>> + scmi: scmi {
>>>> + compatible = "arm,scmi-smc";
>>>> + arm, smc - id = <0x82000002>; <--- Xen management agent channel "smc-id"
>>>> + #address-cells = < 1>;
>>>> + #size-cells = < 0>;
>>>> + #access-controller-cells = < 1>;
>>>> + shmem = <&scmi_shm_0>; <--- Xen management agent channel "shmem"
>>>> +
>>>> + protocol@X{
>>>> + };
>>>> + };
>>>> + };
>>>> + }
>>>> +
>>>> +Important thing to note is that all information about multi-channel support is strictly Xen specific.
>>>> +
>>>> +During initialization the SCMI multi-agent driver uses Host DT SCMI node and
>>>> +"xen,scmi-secondary-agents" property to init itself and then, during Dom0 creation, manipulates
>>>> +Dom0 DT to remove Xen specific SCMI info and update dom0 SCMI nodes with Dom0 SCMI agent specific
>>>> +information.
>>>> +
>>>> +There are two negative points:
>>>> +
>>>> +1) Double DT modification - one is user to set up SCMI Xen support in Host DT, second -
>>>> + Dom0 DT manipulation.
>>>> +2) In case of future support of mailbox shared-memory transport there could be up to 4 mailboxes and
>>>> + up to 2 shared-memories per SCMI agent channel.
>>>> +
>>>> +Hence SCMI multi-agent support is Xen specific knowledge there is a proposal to add it as Xen
>>>> +specific DT definitions and so minimize Host and Dom0 DT manipulations.
>>>> +Those definitions can be added in "/chosen" or, ideally, in "xen,config" node (like in Hyperlaunch design).
>>>> +
>>>> +The SCMI binding stays generic, just two SCMI nodes defined - one for Xen management channel and
>>>> +one for Host Dom0 OSPM.
>>>> +
>>>> +Example of using "chosen" for configuration:
>>>> +
>>>> +.. code::
>>>> +
>>>> + /{
>>>> +
>>>> + chosen {
>>>> + ...
>>>> +
>>>> + // Xen SCMI management channel
>>>> + scmi_shm_0 : sram@47ff0000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
>>>> + };
>>>> + scmi_xen: scmi {
>>>> + compatible = "arm,scmi-smc";
>>>> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
>>>> + #address-cells = < 1>;
>>>> + #size-cells = < 0>;
>>>> + #access-controller-cells = < 1>;
>>>> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
>>>> + };
>>>> +
>>>> + // SCMI multi-agent configuration
>>>> + scmi_shm_2: sram@47ff2000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_3: sram@47ff3000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_4: sram@47ff4000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>>>> + };
>>>> + xen,scmi-secondary-agents = <
>>>> + 1 0x82000003 &scmi_shm
>>>> + 2 0x82000004 &scmi_shm_2
>>>> + 3 0x82000005 &scmi_shm_3
>>>> + 4 0x82000006 &scmi_shm_4>;
>>>> + };
>>>> +
>>>> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
>>>> + scmi_shm: sram@47ff1000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>>>> + };
>>>> +
>>>> + firmware {
>>>> + scmi: scmi {
>>>> + compatible = "arm,scmi-smc";
>>>> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
>>>> + #address-cells = < 1>;
>>>> + #size-cells = < 0>;
>>>> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
>>> By OSPM you mean Dom0 and not Xen? So this is a change compared to a
>>> device tree for baremetal Linux without Xen?
OSPM is OS in general, for example Linux.
>>> Let me ask the same question differently. In the case of barematal Linux
>>> without Xen (no KVM), what would Linux see under /firmware/scmi as
>>> smc-id and shmem? The same as the one that Xen would use for itself? Or
>>> the same as the ones that Dom0 would use when Xen is present?
>> If this DT is used with the baremetal Linux - then the Linux Kernel will
>>
>> see Dom0 "smc-id" and "shmen" under /firmware/scmi.
>>
>>>> + protocol@X{
>>>> + };
>>>> + };
>>>> + };
>>>> + }
>>>> +
>>>> +
>>>> +In the above case:
>>>> +
>>>> +1) Xen SCMI multi-agent can be probed with DT configuration from "chosen" (or special "xen,config")
>>>> + node and all Xen related nodes can be easily dropped from Dom0 DT.
>>>> +2) Host SCMI OSPM channel DT nodes can be copied to Dom0 DT without changes if SCMI enabled for it.
>>>> +3) Future support for mailbox shared-memory transport (SCP SCMI FW) can be simplified as no more
>>>> + manipulation required with Dom0 SCMI "arm,smc-id" and "shmem" DT properties.
>>> Yes, I can see the benefit if we can arrange it so that the underlying
>>> host device tree is the same that Linux would use baremetal. And all the
>>> extra configuration is placed under /chosen in "xen,config" node or
>>> similar. I would probably call it "xen,scmi".
>> Personally, I would keep "xen,config" as it leaves room to add additional
>>
>> configuration nodes in the future.
>>
>>>> +Example of using "xen,config" for configuration:
>>>> +
>>>> +.. code::
>>>> +
>>>> + hypervisor {
>>>> + compatible = “hypervisor,xen”
>>>> +
>>>> + // Configuration container
>>>> + config {
>>>> + compatible = "xen,config";
>>>> + ...
>>>> +
>>>> + // Xen SCMI management channel
>>>> + scmi_shm_0 : sram@47ff0000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
>>>> + };
>>>> + scmi_xen: scmi {
>>>> + compatible = "arm,scmi-smc";
>>>> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
>>>> + #address-cells = < 1>;
>>>> + #size-cells = < 0>;
>>>> + #access-controller-cells = < 1>;
>>>> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
>>>> + };
>>>> +
>>>> + // SCMI multi-agent configuration
>>>> + scmi_shm_2: sram@47ff2000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_3: sram@47ff3000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
>>>> + };
>>>> + scmi_shm_4: sram@47ff4000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
>>>> + };
>>>> + xen,scmi-secondary-agents = <
>>>> + 1 0x82000003 &scmi_shm
>>>> + 2 0x82000004 &scmi_shm_2
>>>> + 3 0x82000005 &scmi_shm_3
>>>> + 4 0x82000006 &scmi_shm_4>;
>>>> + };
>>>> + };
>>>> +
>>>> + /{
>>>> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
>>>> + scmi_shm: sram@47ff1000 {
>>>> + compatible = "arm,scmi-shmem";
>>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
>>>> + };
>>>> +
>>>> + firmware {
>>>> + scmi: scmi {
>>>> + compatible = "arm,scmi-smc";
>>>> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
>>>> + #address-cells = < 1>;
>>>> + #size-cells = < 0>;
>>>> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
>>>> +
>>>> + protocol@X{
>>>> + };
>>>> + };
>>>> + };
>>>> + }
>>>> --
>>>> 2.34.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-18 7:28 ` Julien Grall
@ 2025-06-19 16:15 ` Oleksii Moisieiev
0 siblings, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-19 16:15 UTC (permalink / raw)
To: Julien Grall, xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Bertrand Marquis, Jan Beulich,
Juergen Gross, Michal Orzel, Roger Pau Monné,
Stefano Stabellini, Volodymyr Babchuk, Grygorii Strashko
On 18/06/2025 10:28, Julien Grall wrote:
> Hi Oleksii,
>
> On 19/05/2025 16:50, Oleksii Moisieiev wrote:
>> This patch introduces SCI driver to support for ARM EL3 Trusted
>> Firmware-A
>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
>
> s/multi-agnet/multi-agent/
>
will fix.
>> below.
>>
>> +-----------------------------------------+
>> | |
>> | EL3 TF-A SCMI |
>> +-------+--+-------+--+-------+--+-------++
>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
>> +-----+-+ +---+---+ +--+----+ +---+---+
>> smc-id0 | | | |
>> agent0 | | | |
>> +-----v--------+---------+-----------+----+
>> | | | | |
>> | | | | |
>> +--------------+---------+-----------+----+
>> smc-id1 | smc-id2| smc-idX|
>> agent1 | agent2 | agentX |
>> | | |
>> +----v---+ +--v-----+ +--v-----+
>> | | | | | |
>> | Dom0 | | Dom1 | | DomX |
>> | | | | | |
>> | | | | | |
>> +--------+ +--------+ +--------+
>>
>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC
>> shared
>> memory transport for every Agent in the system.
>>
>> The SCMI Agent transport channel defined by pair:
>> - smc-id: SMC/HVC id used for Doorbell
>> - shmem: shared memory for messages transfer, Xen page aligned,
>> p2m_mmio_direct_nc.
>
> It is not clear why we nention Xen page aligned and
> p2m_mmio_direct_nc. Is this multi-agent protocol tied to Xen?
>
Xen allows mapping only page aligned chunks between domains. Current
implementation supports only page-aligned chunks.
This means that we support only one channel per page, where the shared
memory starts at the beginning of the page.
> That said... p2m_mmio_direct_nc is a type used in the stage 2
> page-tables to indicate how we restrict access from the domain.
>
> The resulting memory attribute will be a combination of stage-1 +
> stage-2. In the future, we may decide to use FWB which will allow Xen
> to force a specific memory attribute.
>
> This is also purely internal decision. In the documentation, you
> should spell out the memory attribute that should be used. From the
> discussion on this patch, it is still unclear whether the region
> should be mapped as Device nGnRE or normal memory non-cacheabl.
>
> Cheers,
>
I will reword this to explicitly mention the correct memory attribute.
Documentation will be updated.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-19 16:15 ` Oleksii Moisieiev
@ 2025-06-22 21:30 ` Stefano Stabellini
2025-06-24 8:42 ` Oleksii Moisieiev
2025-06-23 7:15 ` Jan Beulich
1 sibling, 1 reply; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-22 21:30 UTC (permalink / raw)
To: Oleksii Moisieiev, jbeulich, roger.pau
Cc: Stefano Stabellini, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Juergen Gross, Julien Grall,
Michal Orzel, Volodymyr Babchuk, Grygorii Strashko
On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
> On 18/06/2025 03:04, Stefano Stabellini wrote:
> > On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> >> Hi Stefano,
> >>
> >> I'm very sorry for a long silence. Please see my answers below:
> >>
> >> On 22/05/2025 03:25, Stefano Stabellini wrote:
> >>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> >>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
> >>>>
> >>>> Add chained handling of assigned DT devices to support access-controller
> >>>> functionality through SCI framework, so DT device assign request can be
> >>>> passed to FW for processing and enabling VM access to requested device
> >>>> (for example, device power management through FW interface like SCMI).
> >>>>
> >>>> The SCI access-controller DT device processing is chained after IOMMU
> >>>> processing and expected to be executed for any DT device regardless of its
> >>>> protection by IOMMU (or if IOMMU is disabled).
> >>>>
> >>>> This allows to pass not only IOMMU protected DT device through
> >>>> xl.cfg:"dtdev" property for processing:
> >>>>
> >>>> dtdev = [
> >>>> "/soc/video@e6ef0000", <- IOMMU protected device
> >>>> "/soc/i2c@e6508000", <- not IOMMU protected device
> >>>> ]
> >>>>
> >>>> The change is done in two parts:
> >>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
> >>>> not fail if DT device is not protected by IOMMU
> >>>> 2) add chained call to sci_do_domctl() in do_domctl()
> >>>>
> >>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> >>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >>>> ---
> >>>>
> >>>>
> >>>>
> >>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
> >>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
> >>>> xen/common/domctl.c | 19 +++++++++++++
> >>>> xen/drivers/passthrough/device_tree.c | 6 ++++
> >>>> 4 files changed, 76 insertions(+)
> >>>>
> >>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
> >>>> index e1522e10e2..8efd541c4f 100644
> >>>> --- a/xen/arch/arm/firmware/sci.c
> >>>> +++ b/xen/arch/arm/firmware/sci.c
> >>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
> >>>> return 0;
> >>>> }
> >>>>
> >>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
> >>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> >>>> +{
> >>>> + struct dt_device_node *dev;
> >>>> + int ret = 0;
> >>>> +
> >>>> + switch ( domctl->cmd )
> >>>> + {
> >>>> + case XEN_DOMCTL_assign_device:
> >>>> + ret = -EOPNOTSUPP;
> >>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
> >> The -EOPNOTSUPP code is used because this is part of a chained call after
> >> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
> >> XEN_DOMCTL_assign_device
> >> call is expected to handle any DT device, regardless of whether the DT
> >> device is
> >> protected by an IOMMU or if the IOMMU is disabled.
> >> The following cases are considered:
> >>
> >> 1. IOMMU Protected Device (Success)
> >>
> >> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
> >> we continue
> >> processing the DT device by calling sci_do_domctl.
> >>
> >> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
> >>
> >> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
> >> disabled,
> >> we still proceed to call sci_do_domctl.
> > OK this makes sense. I think it is OK to have a special error code to
> > say "the IOMMU is disabled" but I don't know if it is a good idea to try
> > to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
> > configuration with domctl disabled, for instance.
> >
> > It might be wiser to use a different error code. Maybe ENOENT?
> >
> I see that in the following commit:
>
> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>
> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>
> It's not clear to me why this was done from the commit description.
>
> Maybe we should add commit author?
Roger and Jan might know
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-19 16:15 ` Oleksii Moisieiev
@ 2025-06-22 21:50 ` Stefano Stabellini
0 siblings, 0 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-22 21:50 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: Julien Grall, Stefano Stabellini, Grygorii Strashko,
Bertrand Marquis, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk
[-- Attachment #1: Type: text/plain, Size: 4286 bytes --]
On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
> On 18/06/2025 10:22, Julien Grall wrote:
> > Hi,
> >
> > On 18/06/2025 00:38, Stefano Stabellini wrote:
> >> On Thu, 12 Jun 2025, Grygorii Strashko wrote:
> >>> On 02.06.25 10:17, Bertrand Marquis wrote:
> >>>>> On the other hand, if we also want to handle the case where the SCMI
> >>>>> server could be on a separate co-processor, then what this code is
> >>>>> doing
> >>>>> is not sufficient because we also need a dcache flush, in addition to
> >>>>> the DSB.
> >>>>>
> >>>>> Bertrand, can you double-check?
> >>>>
> >>>> If we want to handle a case where the memory is accessible to a
> >>>> coprocessor
> >>>> but there is no cache coherency, we need to flush the dcache
> >>>> definitely.
> >>>>
> >>>> Seeing the amount of data here, I do agree with Stefano that it
> >>>> would be a
> >>>> good
> >>>> idea to make the provision to flush the data cache in all cases.
> >>>> Even if the
> >>>> data
> >>>> is accessed by a secure partition or the firmware coherently,
> >>>> flushing in
> >>>> all cases
> >>>> would have very limited performance impact here.
> >>>>
> >>>> There is the other solution to have some kind of parameter to say
> >>>> if the
> >>>> accessor
> >>>> has coherent cache access but I do not think the performance impact
> >>>> here
> >>>> would
> >>>> justify such a complexity.
> >>>>
> >>> The SCMI shmem expected to be mapped as MT_NON_CACHEABLE in all cases.
> >
> > I can't find MT_NON_CACHEABLE anywhere in Xen or Linux. My
> > interpretation is that the memory attribute would be normal memory non
> > cacheable. However, this doesn't add up with ...
> >
> Sorry for the confusion. This define was taken from TF-A and it is the
> same as Xen MT_NORMAL_NC.
>
> The main idea was to mention that memory is non_cachable.
>
> >>> The Linux does devm_ioremap() -> ioremap() ->
> >>> (ARM64) __ioremap((addr), (size), __pgprot(PROT_DEVICE_nGnRE))
> >
> > ... this line. This is device nGnRE which is a lot more restrictive
> > (for instance it doesn't allow unaligned access).
> >
> In Xen scmi memory is mapped using ioremap_nocache which is mapped as
> MT_DEVICE_nGnRE (same as linux).
>
> In TF-A SCMI shared memory is mapped as MT_DEVICE (which is
> nGnRE: MAIR_DEV_nGnRE).
>
> Again, sorry for the confusion.
>
> >>>
> >>> There is also note in docs:
> >>> "+- shmem: shared memory for messages transfer, **Xen page aligned**
> >>> with
> >>> mapping``p2m_mmio_direct_nc``."
> >>>
> >>> In the case of SCP - the SCMI shmem can be actually be in SRAM.
> >>>
> >>> So, are you sure cache manipulations are required here?
> >>
> >> No, if the memory is mapped as uncacheable everywhere then the cache
> >> manipulations are not needed. However, we probably still need a dsb.
> >>
> >> I understand now why they decided to use __memcpy_fromio in Linux: it is
> >> not MMIO but they needed a memcpy followed by DSB, so they decided to
> >> reuse the existing MMIO functions although the buffer is not MMIO.
> >
> > From my understanding, memcpy_fromio() is not just a mempcy() + dsb.
> > It also guarantees the access will be aligned (this is not guarantee
> > by our memcpy()).
> >
> From my understanding Linux using memcpy_fromio() because memcpy
> function is highly
>
> optimized in linux and will produce exception,and looking into the
> memcpy implementation
>
> in Xen (xen/arch/arm/arm64/lib/memcpy.S) I'm not sure if it can be used
> instead of memcpy_fromio. Could you please advise how to proceed?
If we map the memory in Xen as normal memory non-cacheable, then for
sure we should be able to use the regular memcpy plus a DSB at the end.
That's because unaligned accesses are allowed.
On the other hand, if we map the memory in Xen as device memory, then we
need to be careful about alignment.
Looking at xen/arch/arm/arm64/lib/memcpy.S, it seems to me that:
- it uses aligned accesses for size >= 16 bytes
- for size < 16 bytes, accesses might be unaligned depending on the
alignment of the start address
- the start address, assuming it is the shared memory start address, is
4K aligned, so we should be fine?
So it seems to me that we should be OK with using the regular memcpy
(plus a DSB at the end). It would be good for someone else to confirm.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-19 16:15 ` [RFC PATCH v4 8/8] docs: armproposa: l " Oleksii Moisieiev
@ 2025-06-22 21:57 ` Stefano Stabellini
2025-06-23 8:02 ` Julien Grall
0 siblings, 1 reply; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-22 21:57 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: Stefano Stabellini, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Julien Grall, Michal Orzel, Roger Pau Monné,
Volodymyr Babchuk, Grygorii Strashko
[-- Attachment #1: Type: text/plain, Size: 22710 bytes --]
On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
> On 18/06/2025 03:35, Stefano Stabellini wrote:
> > On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> >> On 23/05/2025 23:19, Stefano Stabellini wrote:
> >>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> >>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
> >>>>
> >>>> Proposal description to add separate SCMI DT node for Xen management agent
> >>>> under "chosen" or xen-config node, like Hyperlaunch "xen,config".
> >>> I think it is OK to place a larger "xen,config" node under /chosen with
> >>> more information for Xen to setup SCMI more easily.
> >>>
> >>>
> >>>> This proposal introduces a new approach to the Xen multi-domain
> >>>> configuration, where all Xen-specific configuration has been moved
> >>>> under the "/chosen" node. This requires less Dom0 device tree
> >>>> manipulation and isolates Xen configuration from domain configuration.
> >>>>
> >>>> This approach provides the following device tree (DT) parameters:
> >>>>
> >>>> - "xen,scmi-secondary-agents": A Xen-specific parameter under the
> >>>> "/chosen" node, which describes the SCMI agent configuration for
> >>>> the domains.
> >>>> - the SCMI configuration for Xen (privileged agent) and the shared
> >>>> memory configuration for all agents are provided under the "/chosen"
> >>>> node and are used strictly by Xen for its initial configuration.
> >>>> - the scmi_shm and SCMI configuration for Dom0 are placed in the
> >>>> "/firmware/scmi" node so that they can be moved to Dom0 without
> >>>> any changes.
> >>> Isn't the SCMI configuration present in /firmware/scmi referring to the
> >>> privileged agent=0 meant to be used by Xen?
> >>>
> >>> I certainly see benefits in simplifying the configuration and especially
> >>> reducing the amount of changes a user might have to make on the
> >>> underlying device tree, but if the user needs to change /firmware/scmi
> >>> with the Dom0 information, it seems more dangerous and error prone than
> >>> the previous approach.
> >>>
> >> The idea is to move the privileged agent=0 configuration to the /chosen
> >> node and
> >>
> >> assign agent=1 to the Dom0 node under /firmware/scmi.
> >>
> >> Benefits of This Approach:
> >> - No Modification of the Xen DT Node Required
> >>
> >> This eliminates the need to modify the Xen Device Tree (DT) node
> >> before creating Dom0 in
> >>
> >> order to set the correct shared memory (shmem).
> >>
> >> -Consistent SCMI Configuration Format
> >>
> >> The Dom0 DT will have the same SCMI configuration format as other
> >> domains, simplifying the
> >>
> >> overall configuration process.
> >>
> >> - Unified SCMI Configuration Method
> >>
> >> There will no longer be a need to use a different approach for SCMI
> >> configuration in Dom0
> >>
> >> compared to other domains.
> >>
> >> - Separation Between Dom0 and Privileged Node
> >>
> >> This provides a clear separation between the Dom0 node and the
> >> privileged node.
> >>
> >> For example:
> >> If Dom0 only requires the clock protocol, but the Xen SCMI
> >> configuration requires additional protocols,
> >>
> >> this approach allows Dom0 to receive only the necessary
> >> protocol configuration.
> > I don't think this is a good idea because we end up confusing the data
> > for Xen and the data for the DomUs/Dom0 in the host device tree.
> >
> > I think we should follow these very simple guidelines:
> >
> > - The host DTB (the DTB given to Xen at boot) should be the same for Xen
> > and for Linux baremetal (no KVM), with the exception of the data under
> > the /chosen node
> >
> > - We can place Xen specific configurations under the /chosen node in the
> > host DTB, both Xen hypervisor configuration and also Dom0/DomU
> > configurations
> >
> > This way, the host information remains generic and the configuration for
> > Xen the domUs/Dom0 is kept clearly separate from the rest. I don't
> > think we can break these two assumptions but we have more freedom with
> > the rest.
> >
> > If we start with these two simple assumptions, here are the
> > consequences:
> >
> > - data under /firmware/scmi should be the same for Xen and baremetal
> > Linux, ideally it would describe Xen's agent0 channel in both cases
>
> According to the proposal:
> The data under the Host DT /firmware/scmi node will always point to the
> default OSPM agent, which will remain
>
> the same (smc-id and shmem) for both the BSP case (no Xen) and the Xen
> case (Dom0 domain).
>
> Meanwhile, the Xen management agent's SCMI node and configuration are
> expected to be placed under /chosen.
>
> This approach ensures that the Host DT remains as unchanged as possible.
Yes, my main point is that all the device tree information, except for
what is under /chosen, should be left unchanged between the BSP case (no
Xen) and the Xen case.
We have freedom to decide:
- the information we put under /chosen and how to interpret it
- how to use the information under /firmware/scmi when Xen is present
> Currently:
>
> The Host DT /firmware/scmi node requires modification to point to the
> Xen management agent by changing
>
> the smc-id and shmem values.
I don't think we should require changes to /firmware/scmi in the host DT
when Xen is present.
Often, people don't know when or if Xen is present at the time the
Device Tree is generated. So it is best to avoid modification (outside
of /chosen).
> At boot time, during Dom0 creation, the SCMI multi-agent driver reverts
> these changes.
>
>
> > - We can add as many nodes as we like under /chosen, including a
> > xen,config node and also additional nodes for the domains config
> >
> > - We can define the new nodes under /chosen to be as simple as possible
> > for the user to configure them, while also trying to minimize
> > complexity in Xen in terms of DT manipulations
> >
> >
> >
> > If the Xen SCMI configuration data cannot be the same as the Linux
> > baremetal SCMI configuration (i.e. /firmware/scmi has to be different in
> > the two cases) I would still suggest to avoid modifying /firmware/scmi
> > for Xen and instead provide the Xen configuration under /chosen. It is
> > important to keep everything in the host DTB (except /chosen) the same
> > between Linux baremetal and Xen.
> >
> > However, we can add a new node similar to /firmware/scmi under /chosen
> > specifically for Xen, such as /chosen/xen-config/scmi
> >
> > The Dom0 configuration cannot be expected to be under /firmware/scmi.
> > However, it could also be defined under /chosen.
>
> No. The idea is to keep it and provide unchanged, but with possibility
> to "disable SCMI for Dom0"
> - now with Xen bootarg parameter.
> > Keep in mind that the more we add to /chosen the more difficult it will
> > be for the user to configure the system. I think we should plan ahead to
> > have ImageBuilder be able to generate the DT nodes under /chosen for Xen
> > starting from the simplest possible configuration format provided by the
> > user. The more complex and rich are the device tree nodes under /chosen,
> > the more important is the documentation and ImageBuilder support for it.
> >
> >
> Regarding all the other points you’ve mentioned – this is exactly what
> we are trying to achieve
>
> with this proposal.
OK good
> We are proposing the following changes to the approach so that all
> requirements are met, and the
>
> Xen device tree (DT) will remain the same as the Host Platform DT (BSP
> Linux), except for the /chosen node:
>
> ```
>
> /{
>
> chosen {
> ...
>
> // Xen SCMI management channel
> scmi_shm_xen : sram@47ff1000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff1000 0x0 0x1000>;
> };
>
> scmi_xen: scmi {
> compatible = "arm,scmi-smc";
> arm,smc-id = <0x82000003>; <--- Xen manegement agent smc-id
> #address-cells = < 1>;
> #size-cells = < 0>;
> #access-controller-cells = < 1>;
> shmem = <&scmi_shm_xen>; <--- Xen manegement agent shmem
> };
>
> // SCMI multi-agent configuration
> scmi_shm_2: sram@47ff2000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff2000 0x0 0x1000>;
> };
> scmi_shm_3: sram@47ff3000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff3000 0x0 0x1000>;
> };
> scmi_shm_4: sram@47ff4000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff4000 0x0 0x1000>;
> };
>
> xen,scmi-secondary-agents = <
> 0x82000002 &scmi_shm 1
> 0x82000004 &scmi_shm_2 2
> 0x82000005 &scmi_shm_3 3
> 0x82000006 &scmi_shm_4 4>;
> };
>
> // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI
> enabled for it (same address as Host Platform DT)
> scmi_shm: sram@47ff0000 {
> compatible = "arm,scmi-shmem";
> reg = <0x0 0x47ff0000 0x0 0x1000>;
> };
>
> firmware { <-- the below configuration is the same as Host
> Platform DT and will be passed to Dom0
> scmi: scmi {
> compatible = "arm,scmi-smc";
> arm,smc-id = <0x82000002>; <--- Host OSPM agent smc-id
> which is the same as in Host Platform DT
> #address-cells = < 1>;
> #size-cells = < 0>;
> shmem = <&scmi_shm>; <--- Host OSPM agent shmem (same
> address as Host Platform DT)
>
> protocol@X{
> };
> };
> };
> }
>
> ```
I think this is OK, thank you!
> >>>> This configuration allows the use of Xen-specific nodes to provide
> >>>> information strictly needed by Xen while using the default SCMI
> >>>> configuration for Dom0 and other domains. As a result, no additional
> >>>> bindings need to be introduced to the device tree.
> >>> This is not actually implemented by this patch series, right?
> >> It is not. Just posted this document as a proposal.
> >>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> >>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >>>> ---
> >>>>
> >>>>
> >>>>
> >>>> .../arm/firmware/arm-scmi-proposal.rst | 224 ++++++++++++++++++
> >>>> 1 file changed, 224 insertions(+)
> >>>> create mode 100644 docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> >>>>
> >>>> diff --git a/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> >>>> new file mode 100644
> >>>> index 0000000000..fcc2ed2b65
> >>>> --- /dev/null
> >>>> +++ b/docs/hypervisor-guide/arm/firmware/arm-scmi-proposal.rst
> >>>> @@ -0,0 +1,224 @@
> >>>> +
> >>>> +Proposal for SCMI multi-agent driver bindings
> >>>> +=============================================
> >>>> +
> >>>> +Now the Xen configuration for SCMI multi-agent support is done in a bit complicated way, especially
> >>>> +from SCMI multi-agent driver initialization and Dom0 DT manipulation point of view.
> >>>> +Also it does not take into account future requirements to support SCP SCMI FW.
> >>>> +
> >>>> +To enable SCMI multi-agent user need:
> >>>> +
> >>>> +* take host DT with basic SCMI enabled
> >>>> +* add SCMI shared-memory nodes for all agents
> >>>> +* update SCMI node to point on SCMI Xen management channel (``[smc-id, shmem]``)
> >>>> +* add "xen,scmi-secondary-agents" property to the "\chosen" node
> >>>> +
> >>>> +.. code::
> >>>> +
> >>>> + chosen {
> >>>> + xen,scmi-secondary-agents = <
> >>>> + 1 0x82000003 &scmi_shm_1
> >>>> + 2 0x82000004 &scmi_shm_2
> >>>> + 3 0x82000005 &scmi_shm_3
> >>>> + 4 0x82000006 &scmi_shm_4>;
> >>>> + }
> >>>> +
> >>>> + /{
> >>>> + // SCMI shared-memory nodes for all agents
> >>>> + scmi_shm_0 : sram@47ff0000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_1: sram@47ff1000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_2: sram@47ff2000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_3: sram@47ff3000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_4: sram@47ff4000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> >>>> + };
> >>>> +
> >>>> + firmware {
> >>>> + scmi: scmi {
> >>>> + compatible = "arm,scmi-smc";
> >>>> + arm, smc - id = <0x82000002>; <--- Xen management agent channel "smc-id"
> >>>> + #address-cells = < 1>;
> >>>> + #size-cells = < 0>;
> >>>> + #access-controller-cells = < 1>;
> >>>> + shmem = <&scmi_shm_0>; <--- Xen management agent channel "shmem"
> >>>> +
> >>>> + protocol@X{
> >>>> + };
> >>>> + };
> >>>> + };
> >>>> + }
> >>>> +
> >>>> +Important thing to note is that all information about multi-channel support is strictly Xen specific.
> >>>> +
> >>>> +During initialization the SCMI multi-agent driver uses Host DT SCMI node and
> >>>> +"xen,scmi-secondary-agents" property to init itself and then, during Dom0 creation, manipulates
> >>>> +Dom0 DT to remove Xen specific SCMI info and update dom0 SCMI nodes with Dom0 SCMI agent specific
> >>>> +information.
> >>>> +
> >>>> +There are two negative points:
> >>>> +
> >>>> +1) Double DT modification - one is user to set up SCMI Xen support in Host DT, second -
> >>>> + Dom0 DT manipulation.
> >>>> +2) In case of future support of mailbox shared-memory transport there could be up to 4 mailboxes and
> >>>> + up to 2 shared-memories per SCMI agent channel.
> >>>> +
> >>>> +Hence SCMI multi-agent support is Xen specific knowledge there is a proposal to add it as Xen
> >>>> +specific DT definitions and so minimize Host and Dom0 DT manipulations.
> >>>> +Those definitions can be added in "/chosen" or, ideally, in "xen,config" node (like in Hyperlaunch design).
> >>>> +
> >>>> +The SCMI binding stays generic, just two SCMI nodes defined - one for Xen management channel and
> >>>> +one for Host Dom0 OSPM.
> >>>> +
> >>>> +Example of using "chosen" for configuration:
> >>>> +
> >>>> +.. code::
> >>>> +
> >>>> + /{
> >>>> +
> >>>> + chosen {
> >>>> + ...
> >>>> +
> >>>> + // Xen SCMI management channel
> >>>> + scmi_shm_0 : sram@47ff0000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_xen: scmi {
> >>>> + compatible = "arm,scmi-smc";
> >>>> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
> >>>> + #address-cells = < 1>;
> >>>> + #size-cells = < 0>;
> >>>> + #access-controller-cells = < 1>;
> >>>> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
> >>>> + };
> >>>> +
> >>>> + // SCMI multi-agent configuration
> >>>> + scmi_shm_2: sram@47ff2000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_3: sram@47ff3000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_4: sram@47ff4000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> >>>> + };
> >>>> + xen,scmi-secondary-agents = <
> >>>> + 1 0x82000003 &scmi_shm
> >>>> + 2 0x82000004 &scmi_shm_2
> >>>> + 3 0x82000005 &scmi_shm_3
> >>>> + 4 0x82000006 &scmi_shm_4>;
> >>>> + };
> >>>> +
> >>>> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
> >>>> + scmi_shm: sram@47ff1000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> >>>> + };
> >>>> +
> >>>> + firmware {
> >>>> + scmi: scmi {
> >>>> + compatible = "arm,scmi-smc";
> >>>> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
> >>>> + #address-cells = < 1>;
> >>>> + #size-cells = < 0>;
> >>>> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
> >>> By OSPM you mean Dom0 and not Xen? So this is a change compared to a
> >>> device tree for baremetal Linux without Xen?
> OSPM is OS in general, for example Linux.
> >>> Let me ask the same question differently. In the case of barematal Linux
> >>> without Xen (no KVM), what would Linux see under /firmware/scmi as
> >>> smc-id and shmem? The same as the one that Xen would use for itself? Or
> >>> the same as the ones that Dom0 would use when Xen is present?
> >> If this DT is used with the baremetal Linux - then the Linux Kernel will
> >>
> >> see Dom0 "smc-id" and "shmen" under /firmware/scmi.
> >>
> >>>> + protocol@X{
> >>>> + };
> >>>> + };
> >>>> + };
> >>>> + }
> >>>> +
> >>>> +
> >>>> +In the above case:
> >>>> +
> >>>> +1) Xen SCMI multi-agent can be probed with DT configuration from "chosen" (or special "xen,config")
> >>>> + node and all Xen related nodes can be easily dropped from Dom0 DT.
> >>>> +2) Host SCMI OSPM channel DT nodes can be copied to Dom0 DT without changes if SCMI enabled for it.
> >>>> +3) Future support for mailbox shared-memory transport (SCP SCMI FW) can be simplified as no more
> >>>> + manipulation required with Dom0 SCMI "arm,smc-id" and "shmem" DT properties.
> >>> Yes, I can see the benefit if we can arrange it so that the underlying
> >>> host device tree is the same that Linux would use baremetal. And all the
> >>> extra configuration is placed under /chosen in "xen,config" node or
> >>> similar. I would probably call it "xen,scmi".
> >> Personally, I would keep "xen,config" as it leaves room to add additional
> >>
> >> configuration nodes in the future.
> >>
> >>>> +Example of using "xen,config" for configuration:
> >>>> +
> >>>> +.. code::
> >>>> +
> >>>> + hypervisor {
> >>>> + compatible = “hypervisor,xen”
> >>>> +
> >>>> + // Configuration container
> >>>> + config {
> >>>> + compatible = "xen,config";
> >>>> + ...
> >>>> +
> >>>> + // Xen SCMI management channel
> >>>> + scmi_shm_0 : sram@47ff0000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff0000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_xen: scmi {
> >>>> + compatible = "arm,scmi-smc";
> >>>> + arm,smc-id = <0x82000002>; <--- Xen manegement agent smc-id
> >>>> + #address-cells = < 1>;
> >>>> + #size-cells = < 0>;
> >>>> + #access-controller-cells = < 1>;
> >>>> + shmem = <&scmi_shm_0>; <--- Xen manegement agent shmem
> >>>> + };
> >>>> +
> >>>> + // SCMI multi-agent configuration
> >>>> + scmi_shm_2: sram@47ff2000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff2000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_3: sram@47ff3000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff3000 0x0 0x1000>;
> >>>> + };
> >>>> + scmi_shm_4: sram@47ff4000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff4000 0x0 0x1000>;
> >>>> + };
> >>>> + xen,scmi-secondary-agents = <
> >>>> + 1 0x82000003 &scmi_shm
> >>>> + 2 0x82000004 &scmi_shm_2
> >>>> + 3 0x82000005 &scmi_shm_3
> >>>> + 4 0x82000006 &scmi_shm_4>;
> >>>> + };
> >>>> + };
> >>>> +
> >>>> + /{
> >>>> + // Host SCMI OSPM channel - provided to the Dom0 as is if SCMI enabled for it
> >>>> + scmi_shm: sram@47ff1000 {
> >>>> + compatible = "arm,scmi-shmem";
> >>>> + reg = <0x0 0x47ff1000 0x0 0x1000>;
> >>>> + };
> >>>> +
> >>>> + firmware {
> >>>> + scmi: scmi {
> >>>> + compatible = "arm,scmi-smc";
> >>>> + arm,smc-id = <0x82000003>; <--- Host OSPM agent smc-id
> >>>> + #address-cells = < 1>;
> >>>> + #size-cells = < 0>;
> >>>> + shmem = <&scmi_shm>; <--- Host OSPM agent shmem
> >>>> +
> >>>> + protocol@X{
> >>>> + };
> >>>> + };
> >>>> + };
> >>>> + }
> >>>> --
> >>>> 2.34.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-19 16:15 ` Oleksii Moisieiev
@ 2025-06-22 22:15 ` Stefano Stabellini
2025-06-23 8:06 ` Julien Grall
2025-06-25 19:47 ` Oleksii Moisieiev
0 siblings, 2 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-22 22:15 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: Stefano Stabellini, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Julien Grall, Michal Orzel, Roger Pau Monné,
Volodymyr Babchuk, Grygorii Strashko
[-- Attachment #1: Type: text/plain, Size: 25740 bytes --]
On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
> On 18/06/2025 02:22, Stefano Stabellini wrote:
> > On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> >> Hi Stefano,
> >>
> >> I'm very sorry for a long silence. Please see my answers below:
> >>
> >>
> >> On 23/05/2025 23:06, Stefano Stabellini wrote:
> >>> One question for Bertrand below
> >>>
> >>>
> >>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
> >>>> This patch introduces SCI driver to support for ARM EL3 Trusted Firmware-A
> >>>> (TF-A) which provides SCMI interface with multi-agnet support, as shown
> >>>> below.
> >>>>
> >>>> +-----------------------------------------+
> >>>> | |
> >>>> | EL3 TF-A SCMI |
> >>>> +-------+--+-------+--+-------+--+-------++
> >>>> |shmem0 | |shmem1 | |shmem2 | |shmemX |
> >>>> +-----+-+ +---+---+ +--+----+ +---+---+
> >>>> smc-id0 | | | |
> >>>> agent0 | | | |
> >>>> +-----v--------+---------+-----------+----+
> >>>> | | | | |
> >>>> | | | | |
> >>>> +--------------+---------+-----------+----+
> >>>> smc-id1 | smc-id2| smc-idX|
> >>>> agent1 | agent2 | agentX |
> >>>> | | |
> >>>> +----v---+ +--v-----+ +--v-----+
> >>>> | | | | | |
> >>>> | Dom0 | | Dom1 | | DomX |
> >>>> | | | | | |
> >>>> | | | | | |
> >>>> +--------+ +--------+ +--------+
> >>>>
> >>>> The EL3 SCMI multi-agent firmware expected to provide SCMI SMC/HVC shared
> >>>> memory transport for every Agent in the system.
> >>>>
> >>>> The SCMI Agent transport channel defined by pair:
> >>>> - smc-id: SMC/HVC id used for Doorbell
> >>>> - shmem: shared memory for messages transfer, Xen page aligned,
> >>>> p2m_mmio_direct_nc.
> >>>>
> >>>> The follwoing SCMI Agents expected to be defined by SCMI FW to enable SCMI
> >>>> multi-agent functionality under Xen:
> >>>> - Xen manegement agent: trusted agents that accesses to the Base Protocol
> >>>> commands to configure agent specific permissions
> >>>> - OSPM VM agents: non-trusted agent, one for each Guest domain which is
> >>>> allowed direct HW access. At least one OSPM VM agent has to be provided
> >>>> by FW if HW is handled only by Dom0 or Driver Domain.
> >>>>
> >>>> The EL3 SCMI FW expected to implement following Base protocol messages:
> >>>> - BASE_DISCOVER_AGENT
> >>>> - BASE_RESET_AGENT_CONFIGURATION (optional)
> >>>> - BASE_SET_DEVICE_PERMISSIONS (optional)
> >>>>
> >>>> The SCI SCMI SMC multi-agent driver implements following functionality:
> >>>> - It's initialized based on the Host DT SCMI node (only one SCMI interface
> >>>> is supported) which describes Xen management agent SCMI interface.
> >>>>
> >>>> scmi_shm_0 : sram@47ff0000 {
> >>>> compatible = "arm,scmi-shmem";
> >>>> reg = <0x0 0x47ff0000 0x0 0x1000>;
> >>>> };
> >>>> firmware {
> >>>> scmi: scmi {
> >>>> compatible = "arm,scmi-smc";
> >>>> arm, smc - id = <0x82000002>; // Xen manegement agent smc-id
> >>> some extra spaces, it might be a copy/paste error
> >> +
> >>>> \#address-cells = < 1>;
> >>>> \#size-cells = < 0>;
> >>>> \#access-controller - cells = < 1>;
> >>>> shmem = <&scmi_shm_0>; // Xen manegement agent shmem
> >>>>
> >>>> protocol@X{
> >>>> };
> >>>> };
> >>>> };
> >>>>
> >>>> - It obtains Xen specific SCMI Agent's configuration from the Host DT,
> >>>> probes Agents and build SCMI Agents list; The Agents configuration is taken from:
> >>>>
> >>>> chosen {
> >>>> xen,scmi-secondary-agents = <
> >>>> 1 0x82000003 &scmi_shm_1
> >>>> 2 0x82000004 &scmi_shm_2
> >>>> 3 0x82000005 &scmi_shm_3
> >>>> 4 0x82000006 &scmi_shm_4>;
> >>>> }
> >>>>
> >>>> /{
> >>>> scmi_shm_1: sram@47ff1000 {
> >>>> compatible = "arm,scmi-shmem";
> >>>> reg = <0x0 0x47ff1000 0x0 0x1000>;
> >>>> };
> >>>> scmi_shm_2: sram@47ff2000 {
> >>>> compatible = "arm,scmi-shmem";
> >>>> reg = <0x0 0x47ff2000 0x0 0x1000>;
> >>>> };
> >>>> scmi_shm_3: sram@47ff3000 {
> >>>> compatible = "arm,scmi-shmem";
> >>>> reg = <0x0 0x47ff3000 0x0 0x1000>;
> >>>> };
> >>>> }
> >>>> where first item is "agent_id", second - "arm,smc-id", and third - "arm,scmi-shmem" for
> >>>> this agent_id.
> >>>>
> >>>> Note that Xen is the only one entry in the system which need to know
> >>>> about SCMI multi-agent support.
> >>>>
> >>>> - It implements the SCI subsystem interface required for configuring and
> >>>> enabling SCMI functionality for Dom0/hwdom and Guest domains. To enable
> >>>> SCMI functionality for domain it has to be configured with unique supported
> >>>> SCMI Agent_id and use corresponding SCMI SMC/HVC shared memory transport
> >>>> [smc-id, shmem] defined for this SCMI Agent_id.
> >>>> - Once Xen domain is configured it can communicate with EL3 SCMI FW:
> >>>> -- zero-copy, the guest domain puts SCMI message in shmem;
> >>>> -- the guest triggers SMC/HVC exception with smc-id (doorbell);
> >>>> -- the Xen driver catches exception, do checks and synchronously forwards
> >>>> it to EL3 FW.
> >>>> - the Xen driver sends BASE_RESET_AGENT_CONFIGURATION message to Xen
> >>>> management agent channel on domain destroy event. This allows to reset
> >>>> resources used by domain and so implement use-case like domain reboot.
> >>>>
> >>>> Dom0 Enable SCMI SMC:
> >>>> - pass dom0_scmi_agent_id=<agent_id> in Xen command line. if not provided
> >>>> SCMI will be disabled for Dom0 and all SCMI nodes removed from Dom0 DT.
> >>>> The driver updates Dom0 DT SCMI node "arm,smc-id" value and fix up shmem
> >>>> node according to assigned agent_id.
> >>>>
> >>>> Guest domains enable SCMI SMC:
> >>>> - xl.cfg: add configuration option as below
> >>>>
> >>>> arm_sci = "type=scmi_smc_multiagent,agent_id=2"
> >>>>
> >>>> - xl.cfg: enable access to the "arm,scmi-shmem" which should correspond assigned agent_id for
> >>>> the domain, for example:
> >>>>
> >>>> iomem = [
> >>>> "47ff2,1@22001",
> >>>> ]
> >>> Looking at the code and the configuration options, it looks like it is
> >>> possible to map a scmi-shmem channel at a different address for the
> >>> guest. It seems like it would work. Is that correct?
> >>>
> >> Yes it will. in our case address 22001000 should be the save as
> >> sram@22001000 in the domain device-tree.
> >>>> - DT: add SCMI nodes to the Driver domain partial device tree as in the
> >>>> below example. The "arm,smc-id" should correspond assigned agent_id for the domain:
> >>>>
> >>>> passthrough {
> >>>> scmi_shm_0: sram@22001000 {
> >>>> compatible = "arm,scmi-shmem";
> >>>> reg = <0x0 0x22001000 0x0 0x1000>;
> >>>> };
> >>>>
> >>>> firmware {
> >>>> compatible = "simple-bus";
> >>>> scmi: scmi {
> >>>> compatible = "arm,scmi-smc";
> >>>> arm,smc-id = <0x82000004>;
> >>>> shmem = <&scmi_shm_0>;
> >>>> ...
> >>>> }
> >>>> }
> >>>> }
> >>>>
> >>>> SCMI "4.2.1.1 Device specific access control"
> >>>>
> >>>> The XEN SCI SCMI SMC multi-agent driver performs "access-controller" provider function
> >>>> in case EL3 SCMI FW implements SCMI "4.2.1.1 Device specific access control" and provides the
> >>>> BASE_SET_DEVICE_PERMISSIONS command to configure the devices that an agents have access to.
> >>>> The DT SCMI node should "#access-controller-cells=<1>" property and DT devices should be bound
> >>>> to the Xen SCMI.
> >>>>
> >>>> &i2c1 {
> >>>> access-controllers = <&scmi 0>;
> >>>> };
> >>>>
> >>>> The Dom0 and dom0less domains DT devices will be processed automatically through
> >>>> sci_assign_dt_device() call, but to assign SCMI devices from toolstack the xl.cfg:"dtdev" property
> >>>> shell be used:
> >>>>
> >>>> dtdev = [
> >>>> "/soc/i2c@e6508000",
> >>>> ]
> >>>>
> >>>> xl.cfg:dtdev will contain all nodes which are under SCMI management (not only those which are behind IOMMU).
> >>>>
> >>>> [1]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/firmware/arm,scmi.yaml
> >>>> [2]https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/access-controllers/access-controllers.yaml
> >>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
> >>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
> >>> Thanks for the long explanation, great work! I am really looking forward
> >>> to have this feature in the tree soon.
> >>>
> >>>
> >>>> ---
> >>>>
> >>>> Changes in v4:
> >>>> - toolstack comments from Anthony PERARD
> >>>> - added dom0less support
> >>>> - added doc for "xen,scmi-secondary-agents"
> >>>>
> >>>> docs/man/xl.cfg.5.pod.in | 13 +
> >>>> docs/misc/arm/device-tree/booting.txt | 60 ++
> >>>> docs/misc/xen-command-line.pandoc | 9 +
> >>>> tools/libs/light/libxl_arm.c | 4 +
> >>>> tools/libs/light/libxl_types.idl | 4 +-
> >>>> tools/xl/xl_parse.c | 12 +
> >>>> xen/arch/arm/dom0less-build.c | 11 +
> >>>> xen/arch/arm/domain_build.c | 3 +-
> >>>> xen/arch/arm/firmware/Kconfig | 11 +
> >>>> xen/arch/arm/firmware/Makefile | 1 +
> >>>> xen/arch/arm/firmware/scmi-proto.h | 164 ++++
> >>>> xen/arch/arm/firmware/scmi-shmem.c | 173 ++++
> >>>> xen/arch/arm/firmware/scmi-shmem.h | 45 +
> >>>> xen/arch/arm/firmware/scmi-smc-multiagent.c | 860 ++++++++++++++++++++
> >>>> xen/include/public/arch-arm.h | 3 +
> >>>> 15 files changed, 1371 insertions(+), 2 deletions(-)
> >>>> create mode 100644 xen/arch/arm/firmware/scmi-proto.h
> >>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.c
> >>>> create mode 100644 xen/arch/arm/firmware/scmi-shmem.h
> >>>> create mode 100644 xen/arch/arm/firmware/scmi-smc-multiagent.c
> >>>>
> >>>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> >>>> index 1ccf50b8ea..302c46d8bc 100644
> >>>> --- a/docs/man/xl.cfg.5.pod.in
> >>>> +++ b/docs/man/xl.cfg.5.pod.in
> >>>> @@ -3122,8 +3122,21 @@ single SCMI OSPM agent support.
> >>>> Should be used together with B<dom0_scmi_smc_passthrough> Xen command line
> >>>> option.
> >>>>
> >>>> +=item B<scmi_smc_multiagent>
> >>>> +
> >>>> +Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> >>>> +SMC calls forwarding from domain to the EL3 firmware (like Trusted Firmware-A)
> >>>> +with a multi SCMI OSPM agent support. The SCMI B<agent_id> should be
> >>>> +specified for the guest.
> >>>> +
> >>>> =back
> >>>>
> >>>> +=item B<agent_id=NUMBER>
> >>>> +
> >>>> +Specifies a non-zero ARM SCI agent id for the guest. This option is mandatory
> >>>> +if the SCMI SMC support is enabled for the guest. The agent ids of domains
> >>>> +existing on a single host must be unique and in the range [1..255].
> >>>> +
> >>>> =back
> >>>>
> >>>> =back
> >>>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> >>>> index 8943c04173..c8923ab8b2 100644
> >>>> --- a/docs/misc/arm/device-tree/booting.txt
> >>>> +++ b/docs/misc/arm/device-tree/booting.txt
> >>>> @@ -296,6 +296,20 @@ with the following properties:
> >>>> Should be used together with dom0_scmi_smc_passthrough Xen command line
> >>>> option.
> >>>>
> >>>> + - "scmi_smc_multiagent"
> >>>> +
> >>>> + Enables ARM SCMI SMC multi-agent support for the guest by enabling SCMI over
> >>>> + SMC calls forwarding from domain to the EL3 firmware (like ARM
> >>>> + Trusted Firmware-A) with a multi SCMI OSPM agent support.
> >>>> + The SCMI agent_id should be specified for the guest with "xen,sci_agent_id"
> >>>> + property.
> >>>> +
> >>>> +- "xen,sci_agent_id"
> >>>> +
> >>>> + Specifies a non-zero ARM SCI agent id for the guest. This option is
> >>>> + mandatory if the SCMI SMC "scmi_smc_multiagent" support is enabled for
> >>>> + the guest. The agent ids of guest must be unique and in the range [1..255].
> >>>> +
> >>>> Under the "xen,domain" compatible node, one or more sub-nodes are present
> >>>> for the DomU kernel and ramdisk.
> >>>>
> >>>> @@ -764,3 +778,49 @@ The automatically allocated static shared memory will get mapped at
> >>>> 0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
> >>>> guest physical address space. DomU1 is explicitly defined as the owner domain,
> >>>> and DomU2 is the borrower domain.
> >>>> +
> >>>> +SCMI SMC multi-agent support
> >>>> +============================
> >>>> +
> >>>> +For enabling the ARM SCMI SMC multi-agent support (enabled by CONFIG_SCMI_SMC_MA)
> >>>> +the Xen specific SCMI Agent's configuration shell be provided in the Host DT
> >>>> +according to the SCMI compliant EL3 Firmware specification with
> >>>> +ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
> >>>> +the top-level "chosen" node:
> >>>> +
> >>>> +- xen,scmi-secondary-agents
> >>>> +
> >>>> + Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
> >>>> + available for Xen. Each Agent defined as triple consisting of:
> >>>> + SCMI agent_id,
> >>>> + SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
> >>>> + phandle to SCMI SHM assigned for the agent transport ("arm,scmi-shmem").
> >>>> +
> >>>> +As an example:
> >>>> +
> >>>> +chosen {
> >>>> + xen,scmi-secondary-agents = <
> >>>> + 1 0x82000003 &scmi_shm_1
> >>>> + 2 0x82000004 &scmi_shm_2
> >>>> + 3 0x82000005 &scmi_shm_3
> >>>> + 4 0x82000006 &scmi_shm_4>;
> >>>> +}
> >>> NIT: it should be };
> >> +
> >>> Looking at scmi_probe, collect_agents, and the following SCMI
> >>> SCMI_BASE_DISCOVER_AGENT request, I wonder: do we actually need this
> >>> information?
> >>>
> >>> It looks like we can discover the agend_ids for every channel, I guess
> >>> what we need to know is the shmem location for every channel? But the
> >>> full list of shmem channel is available below thanks to the scmi-shmem
> >>> nodes.
> >>>
> >>> So, we have the list of scmi-shmem anyway, and we can probe the
> >>> agent_id. The only parameter left is the smc_id/func_id.
> >>>
> >>> Or maybe smc_id/func_id can be calculated from agent_id?
> >>>
> >>> I am asking mostly because if a user is supposed to add this
> >>> xen,scmi-secondary-agents property, where are they supposed to find the
> >>> smc_id/func_id information?
> >>>
> >>> It is important that we write down in this document how the user is
> >>> expected to find out what 1 is 0x82000003 which is scmi_shm_1.
> >> That's a very good question! The issue here is that there are no
> >>
> >> explicit requirements defining the relationship between agent_id and
> >> func_id.
> >>
> >>
> >> For example, in ARM-TF, different implementations can use different
> >> func_ids.
> >>
> >> To provide better flexibility, we decided to separate agent_id from func_id.
> >>
> >>
> >> Currently, the SCMI_BASE_DISCOVER_AGENT calls from the probe are intended to
> >>
> >> verify that all registered agents are present and that the configuration
> >> is correct.
> >>
> >> However, I understand that this additional validation could be optional
> >> to save traffic.
> >>
> >>
> >> To address this, I’m considering adding a configuration option, such as
> >>
> >> CONFIG_SCMI_AGENT_VALIDATION, which can be disabled to reduce boot time
> >> if this
> >>
> >> validation is not necessary for certain use cases.
> >>
> >>
> >> Lastly, I’ll be updating the document to include clearer information
> >> about the
> >>
> >> relationship between func_id and agent_id in the upcoming v5.
> >
> > The key point here is to make it easier for the user. If we can make
> > agent_id or func_id optional it would make users lives easier.
> >
> > Alternative, or in addition to this, we should make the docs as clear as
> > possible so that people can figure it out without having to ask
> > questions on xen-devel.
> >
> >
>
> [OM ANSWER]
>
> SCMI_BASE_DISCOVER_AGENT message requires agent_id. I think we can calculate
>
> agent_id from func_id, the documentation will look like this:
>
> ```
>
> For enabling the ARM SCMI SMC multi-agent support (enabled by
> CONFIG_SCMI_SMC_MA)
> the Xen specific SCMI Agent's configuration shell be provided in the Host DT
> according to the SCMI compliant EL3 Firmware specification with
> ARM SMC/HVC transport using property "xen,scmi-secondary-agents" under
> the top-level "chosen" node:
>
> - xen,scmi-secondary-agents
>
> Defines a set of SCMI agents configuration supported by SCMI EL3 FW and
> available for Xen. Each Agent defined as triple consisting of:
> SMC/HVC function_id assigned for the agent transport ("arm,smc-id"),
> phandle to SCMI SHM assigned for the agent transport
> ("arm,scmi-shmem"),
>
> SCMI agent_id (optional)
>
> As an example:
>
> chosen {
> xen,scmi-secondary-agents = <
> 0x82000003 &scmi_shm_1 1
> 0x82000004 &scmi_shm_2 2
> 0x82000005 &scmi_shm_3 3
> 0x82000006 &scmi_shm_4 4>;
> }
>
> or
>
> chosen {
> xen,scmi-secondary-agents = <
> 0x82000003 &scmi_shm_1
> 0x82000004 &scmi_shm_2
> 0x82000005 &scmi_shm_3
> 0x82000006 &scmi_shm_4>;
> }
>
> If the optional parameter agent_id is not provided, then Xen will
> request agent_id for each registered
>
> channel from the firmware.
>
> This procedure requires sending BASE_DISCOVER_AGENT using func_id and
> shmem region for each
>
> registered agent and discover it's agent_id. This operation requires all
> memory reserved for the agents
>
> to be mapped and then unmapped which can cause boot time increasing. So
> if boot time is important it is
>
> preferable to provide agent_in in the device-tree.
>
> There are some cases when Xen may not have an access to other agent
> memory, including scmi shmem due to
>
> a very strict FFI configuration. In this case agent_id should be
> provided inside xem,scmi-secondary-agents parameter.
>
> ```
>
> What do you think about this approach?
This is better, thank you!
> I will add information about all point into the docs and will try to
> make it as clear as possible.
[...]
> >>> I don't understand why we need __memcpy_fromio and __memcpy_toio: can't
> >>> we use a simple memcpy?
> >> This approach was used because we're trying to access shared memory
> >> between two independent systems:
> >>
> >> Arm-TF and Xen in our case which places some chunks of data to the same
> >> memory. And, according to the [0]
> >>
> >> ```
> >>
> >> Some devices (such as framebuffers) would like to use larger transfers than
> >> 8 bytes at a time. For these devices, the memcpy_toio(),
> >> memcpy_fromio() and memset_io() functions are
> >> provided. Do not use memset or memcpy on IO addresses; they are not
> >> guaranteed to copy data in order.
> >>
> >> ```
> >>
> >> Also, the same approach was used by Arm team when introducing scmi
> >> driver to the Linux kernel [1]
> >>
> >>
> >> [0]:https://www.kernel.org/doc/Documentation/driver-api/device-io.rst
> >>
> >> [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
> > Keep in mind that [0] refers specifically to access to MMIO regions. I
> > assume that the SCMI shared buffers are on normal memory? Regarding [1],
> > it makes sense if Linux is trying to support shared memory over MMIO.
> >
> > Looking at one of your replies below, I am guessing the memory buffers
> > are actually in normal memory but the issue is that TF-A is mapping them
> > as uncacheable. Is that correct?
> >
> > In that case, I still don't understand why a simple memcpy would not be
> > sufficient. Can you check?
> >
> > If yes, then for now I would just simplify it down to memcpy. When
> > someone adds support for an SCMI server elsewhere we could look into
> > adding a more sophisticated memcpy and we can look at the details at
> > that point in time. Specifically, I am not convinced that memcpy_toio
> > and memcpy_fromio would work if the SCMI server is on a separate
> > non-coherent microcontroller.
> >
> According to the TF-A implementation SCMI memory
>
> is mapped with the flags: MT_DEVICE (like for stm32mp1) or
> MT_NON_CACHEABLE (for rpi3)
>
> So probably you're right. I will check with simple memcpy.
There is a difference between MT_DEVICE and MT_NON_CACHEABLE: as far as
I know MT_DEVICE requires aligned accesses while MT_NON_CACHEABLE does
not.
However, as I wrote in the other email, if I am not mistaken the current
implementation of memcpy might work well for us anyway. (To be
confirmed.)
[...]
> >>> Are you sure it is worth to go through all this trouble to modify FDT in
> >>> place when we could simply generate the DT node from scratch like we do
> >>> for example for the GIC? This seems to be more error prone as well. Is
> >>> generating it from scratch is really difficult? If it is difficult then OK.
> >>>
> >> In the last patch [0] of the series, there is a proposal to separate
> >>
> >> the Xen privileged agent from the Dom0 agent. This eliminates the
> >>
> >> need to modify the Xen Device Tree Blob (DTB), which is a positive
> >> improvement
> >>
> >> since the Dom0 agent node is already present in the device tree.
> >>
> >> However, the issue with SCMI node generation lies in the need to handle
> >>
> >> the list of protocols that are supported across the system.
> >>
> >> If we want to generate the SCMI node for Dom0, we need to:
> >>
> >> Copy these protocols from a centralized or predefined source.
> >> Set the correct phandle for each subnode within the SCMI node, ensuring
> >>
> >> accurate representation and functionality.
> >> This extra step of managing the protocols and phandles adds
> >>
> >> complexity but is necessary for ensuring proper support for Dom0.
> >>
> >> [0]
> >> https://lists.xenproject.org/archives/html/xen-devel/2025-05/msg01041.html
> > I was only commenting that rather than trying to modify the DT in place
> > we could create the node for Dom0 from scratch (artificially), based on
> > host DT information as required (fetching data from the host DT as
> > required and copying it to the Dom0 DT).
> >
> The problem is that scmi node, apart from the default parameters has a
> list of
>
> supported protocols which may be different for Xen and Dom0 so to
> generate Dom0 node
>
> from scratch we need to copy(or even generate) these nodes which will
> require complex implementation.
>
> In contrary, if we go with implementation, proposed in the last patch -
> then we can have
>
> more cleaner Xen device tree, which will look like this:
>
> ```
>
> chosen {
>
> scmi_xen: scmi {
>
> arm,smc-id = <0x82000002>;
>
> shmem= <$shm_0>;
>
> ... # no protocols description here
>
> };
>
> firmware {
>
> scmi {
>
> arm,smc-id = <0x82000003>;
>
> shmem= <$shm_1>;
>
> protocol@X{
>
> };
>
> }
>
> };
>
> ```
>
> In this case, we don’t provide any protocol information for the Xen
> agent and instead set protocols for Dom0.
>
> This ensures that each node has the information it requires.
Leaving aside how we generate the device tree nodes for Dom0, which is
of minor importance, we have two potentially conflicting goals:
1) no Xen-specific changes to Device Tree outside of /chosen
2) When Xen is present, Xen should be the one using the information in the Host Device Tree
With 1), I mean that the Host Device Tree, outside of the /chosen node,
should be the same for the BSP (no Xen) and for Xen. That is because it
should describe the platform, no matter the software running on it. Also
at the time the Device Tree is generated, often it is not known if Xen
will be running on the platform or not.
With 2) I meant that ideally when Xen is present, Xen should be the one
to use the information under /firmware/scmi, because Xen is the
principal SCMI client. Then Xen would generate somehow the SCMI device
tree nodes for Dom0 and the DomUs.
1) is more important than 2).
From what you wrote, it looks like the SCMI device tree nodes are
different between Linux baremetal and Xen. If /firmware/scmi describes
what Linux baremetal should use, then it would work for Linux baremetal
but it would *not* work for Xen's own SCMI connection. Is that correct?
If that is the case, that is unfortunate. We need to compromise on 2).
I would keep /firmware/scmi as the one used by Linux baremetal because I
think we should uphold 1) above all else.
Then the Xen SCMI nodes could be placed under /chosen. This is less than
ideal because /chosen should contain Xen/Linux configurations, not
firmware interfaces. But it is the best we can do in a difficult
situation.
I think that whether the Dom0 device tree nodes are artificially
generated by Xen, or copied from the Host Device Tree somewhere, it
matters less.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 21:30 ` Stefano Stabellini
@ 2025-06-23 7:15 ` Jan Beulich
2025-06-23 7:28 ` Oleksii Moisieiev
2025-06-25 19:56 ` Oleksii Moisieiev
1 sibling, 2 replies; 67+ messages in thread
From: Jan Beulich @ 2025-06-23 7:15 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko,
Stefano Stabellini
On 19.06.2025 18:15, Oleksii Moisieiev wrote:
>
> On 18/06/2025 03:04, Stefano Stabellini wrote:
>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>> Hi Stefano,
>>>
>>> I'm very sorry for a long silence. Please see my answers below:
>>>
>>> On 22/05/2025 03:25, Stefano Stabellini wrote:
>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>
>>>>> Add chained handling of assigned DT devices to support access-controller
>>>>> functionality through SCI framework, so DT device assign request can be
>>>>> passed to FW for processing and enabling VM access to requested device
>>>>> (for example, device power management through FW interface like SCMI).
>>>>>
>>>>> The SCI access-controller DT device processing is chained after IOMMU
>>>>> processing and expected to be executed for any DT device regardless of its
>>>>> protection by IOMMU (or if IOMMU is disabled).
>>>>>
>>>>> This allows to pass not only IOMMU protected DT device through
>>>>> xl.cfg:"dtdev" property for processing:
>>>>>
>>>>> dtdev = [
>>>>> "/soc/video@e6ef0000", <- IOMMU protected device
>>>>> "/soc/i2c@e6508000", <- not IOMMU protected device
>>>>> ]
>>>>>
>>>>> The change is done in two parts:
>>>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>>>>> not fail if DT device is not protected by IOMMU
>>>>> 2) add chained call to sci_do_domctl() in do_domctl()
>>>>>
>>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>>> ---
>>>>>
>>>>>
>>>>>
>>>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>>>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>>>>> xen/common/domctl.c | 19 +++++++++++++
>>>>> xen/drivers/passthrough/device_tree.c | 6 ++++
>>>>> 4 files changed, 76 insertions(+)
>>>>>
>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>> index e1522e10e2..8efd541c4f 100644
>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>> return 0;
>>>>> }
>>>>>
>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>> +{
>>>>> + struct dt_device_node *dev;
>>>>> + int ret = 0;
>>>>> +
>>>>> + switch ( domctl->cmd )
>>>>> + {
>>>>> + case XEN_DOMCTL_assign_device:
>>>>> + ret = -EOPNOTSUPP;
>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>> XEN_DOMCTL_assign_device
>>> call is expected to handle any DT device, regardless of whether the DT
>>> device is
>>> protected by an IOMMU or if the IOMMU is disabled.
>>> The following cases are considered:
>>>
>>> 1. IOMMU Protected Device (Success)
>>>
>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>> we continue
>>> processing the DT device by calling sci_do_domctl.
>>>
>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>
>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>> disabled,
>>> we still proceed to call sci_do_domctl.
>> OK this makes sense. I think it is OK to have a special error code to
>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>> configuration with domctl disabled, for instance.
>>
>> It might be wiser to use a different error code. Maybe ENOENT?
>>
> I see that in the following commit:
>
> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>
> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>
> It's not clear to me why this was done from the commit description.
This has been discussed many times elsewhere. Many of our ENOSYS uses are
simply wrong. ENOSYS has very limited applicability: Unavailability of a
top-level hypercall (originally: syscall).
> Maybe we should add commit author?
You might, but Paul hasn't been active in Xen for quite some time now.
Jan
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-23 7:15 ` Jan Beulich
@ 2025-06-23 7:28 ` Oleksii Moisieiev
2025-06-25 19:56 ` Oleksii Moisieiev
1 sibling, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-23 7:28 UTC (permalink / raw)
To: Jan Beulich
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko,
Stefano Stabellini, paul.durrant@citrix.com
Adding Paul to the conversation.
On 23/06/2025 10:15, Jan Beulich wrote:
> On 19.06.2025 18:15, Oleksii Moisieiev wrote:
>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>> Hi Stefano,
>>>>
>>>> I'm very sorry for a long silence. Please see my answers below:
>>>>
>>>> On 22/05/2025 03:25, Stefano Stabellini wrote:
>>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>>
>>>>>> Add chained handling of assigned DT devices to support access-controller
>>>>>> functionality through SCI framework, so DT device assign request can be
>>>>>> passed to FW for processing and enabling VM access to requested device
>>>>>> (for example, device power management through FW interface like SCMI).
>>>>>>
>>>>>> The SCI access-controller DT device processing is chained after IOMMU
>>>>>> processing and expected to be executed for any DT device regardless of its
>>>>>> protection by IOMMU (or if IOMMU is disabled).
>>>>>>
>>>>>> This allows to pass not only IOMMU protected DT device through
>>>>>> xl.cfg:"dtdev" property for processing:
>>>>>>
>>>>>> dtdev = [
>>>>>> "/soc/video@e6ef0000", <- IOMMU protected device
>>>>>> "/soc/i2c@e6508000", <- not IOMMU protected device
>>>>>> ]
>>>>>>
>>>>>> The change is done in two parts:
>>>>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>>>>>> not fail if DT device is not protected by IOMMU
>>>>>> 2) add chained call to sci_do_domctl() in do_domctl()
>>>>>>
>>>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>>>> ---
>>>>>>
>>>>>>
>>>>>>
>>>>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>>>>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>>>>>> xen/common/domctl.c | 19 +++++++++++++
>>>>>> xen/drivers/passthrough/device_tree.c | 6 ++++
>>>>>> 4 files changed, 76 insertions(+)
>>>>>>
>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>> +{
>>>>>> + struct dt_device_node *dev;
>>>>>> + int ret = 0;
>>>>>> +
>>>>>> + switch ( domctl->cmd )
>>>>>> + {
>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>> + ret = -EOPNOTSUPP;
>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>> XEN_DOMCTL_assign_device
>>>> call is expected to handle any DT device, regardless of whether the DT
>>>> device is
>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>> The following cases are considered:
>>>>
>>>> 1. IOMMU Protected Device (Success)
>>>>
>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>> we continue
>>>> processing the DT device by calling sci_do_domctl.
>>>>
>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>
>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>> disabled,
>>>> we still proceed to call sci_do_domctl.
>>> OK this makes sense. I think it is OK to have a special error code to
>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>> configuration with domctl disabled, for instance.
>>>
>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>
>> I see that in the following commit:
>>
>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>
>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>
>> It's not clear to me why this was done from the commit description.
> This has been discussed many times elsewhere. Many of our ENOSYS uses are
> simply wrong. ENOSYS has very limited applicability: Unavailability of a
> top-level hypercall (originally: syscall).
>
>> Maybe we should add commit author?
> You might, but Paul hasn't been active in Xen for quite some time now.
>
> Jan
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-22 21:57 ` Stefano Stabellini
@ 2025-06-23 8:02 ` Julien Grall
2025-06-25 19:47 ` Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-23 8:02 UTC (permalink / raw)
To: Stefano Stabellini, Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
Hi Stefano and Oleksii,
Let me start with a bit of process. This is discussion is getting fairly
difficult to follow.... Can you please trim unrelevant bits when replying?
On 22/06/2025 22:57, Stefano Stabellini wrote:
> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>> On 18/06/2025 03:35, Stefano Stabellini wrote:
>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>> On 23/05/2025 23:19, Stefano Stabellini wrote:
>>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>> the same (smc-id and shmem) for both the BSP case (no Xen) and the Xen
>> case (Dom0 domain).
>>
>> Meanwhile, the Xen management agent's SCMI node and configuration are
>> expected to be placed under /chosen.
>>
>> This approach ensures that the Host DT remains as unchanged as possible.
>
> Yes, my main point is that all the device tree information, except for
> what is under /chosen, should be left unchanged between the BSP case (no
> Xen) and the Xen case.
>
> We have freedom to decide:
> - the information we put under /chosen and how to interpret it
> - how to use the information under /firmware/scmi when Xen is present
>
>
>> Currently:
>>
>> The Host DT /firmware/scmi node requires modification to point to the
>> Xen management agent by changing
>>
>> the smc-id and shmem values.
>
> I don't think we should require changes to /firmware/scmi in the host DT
> when Xen is present.
>
> Often, people don't know when or if Xen is present at the time the
> Device Tree is generated. So it is best to avoid modification (outside
> of /chosen).
I am probably missing something. But it looks like TF-A requires to
suport multi-agent so Xen can use it. Am I correct?
Furthermore, I can't tell why the multi-agent support is Xen specific.
Surely, you may want something similar with other hypervisors? If not,
then my next question is why does Xen needs to do things differently?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-22 22:15 ` Stefano Stabellini
@ 2025-06-23 8:06 ` Julien Grall
2025-06-23 19:27 ` Stefano Stabellini
2025-06-25 19:47 ` Oleksii Moisieiev
1 sibling, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-23 8:06 UTC (permalink / raw)
To: Stefano Stabellini, Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
Hi Stefano,
On 22/06/2025 23:15, Stefano Stabellini wrote:
> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>> On 18/06/2025 02:22, Stefano Stabellini wrote:
>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>> [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
>>> Keep in mind that [0] refers specifically to access to MMIO regions. I
>>> assume that the SCMI shared buffers are on normal memory? Regarding [1],
>>> it makes sense if Linux is trying to support shared memory over MMIO.
>>>
>>> Looking at one of your replies below, I am guessing the memory buffers
>>> are actually in normal memory but the issue is that TF-A is mapping them
>>> as uncacheable. Is that correct?
>>>
>>> In that case, I still don't understand why a simple memcpy would not be
>>> sufficient. Can you check?
>>>
>>> If yes, then for now I would just simplify it down to memcpy. When
>>> someone adds support for an SCMI server elsewhere we could look into
>>> adding a more sophisticated memcpy and we can look at the details at
>>> that point in time. Specifically, I am not convinced that memcpy_toio
>>> and memcpy_fromio would work if the SCMI server is on a separate
>>> non-coherent microcontroller.
>>>
>> According to the TF-A implementation SCMI memory
>>
>> is mapped with the flags: MT_DEVICE (like for stm32mp1) or
>> MT_NON_CACHEABLE (for rpi3)
>>
>> So probably you're right. I will check with simple memcpy.
>
> There is a difference between MT_DEVICE and MT_NON_CACHEABLE: as far as
> I know MT_DEVICE requires aligned accesses while MT_NON_CACHEABLE does
> not.
>
> However, as I wrote in the other email, if I am not mistaken the current
> implementation of memcpy might work well for us anyway. (To be
> confirmed.)
I am not entirely sure what exactly you want to confirm. I have already
mentioned several time that our memcpy() on arm64 is using unaligned
access. So it can't be used for copying data to/from device memory area.
Are you looking for a different things?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-23 8:06 ` Julien Grall
@ 2025-06-23 19:27 ` Stefano Stabellini
2025-06-23 20:38 ` Julien Grall
0 siblings, 1 reply; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-23 19:27 UTC (permalink / raw)
To: Julien Grall
Cc: Stefano Stabellini, Oleksii Moisieiev,
xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On Mon, 23 Jun 2025, Julien Grall wrote:
> Hi Stefano,
>
> On 22/06/2025 23:15, Stefano Stabellini wrote:
> > On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
> > > On 18/06/2025 02:22, Stefano Stabellini wrote:
> > > > On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> > > > > [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
> > > > Keep in mind that [0] refers specifically to access to MMIO regions. I
> > > > assume that the SCMI shared buffers are on normal memory? Regarding [1],
> > > > it makes sense if Linux is trying to support shared memory over MMIO.
> > > >
> > > > Looking at one of your replies below, I am guessing the memory buffers
> > > > are actually in normal memory but the issue is that TF-A is mapping them
> > > > as uncacheable. Is that correct?
> > > >
> > > > In that case, I still don't understand why a simple memcpy would not be
> > > > sufficient. Can you check?
> > > >
> > > > If yes, then for now I would just simplify it down to memcpy. When
> > > > someone adds support for an SCMI server elsewhere we could look into
> > > > adding a more sophisticated memcpy and we can look at the details at
> > > > that point in time. Specifically, I am not convinced that memcpy_toio
> > > > and memcpy_fromio would work if the SCMI server is on a separate
> > > > non-coherent microcontroller.
> > > >
> > > According to the TF-A implementation SCMI memory
> > >
> > > is mapped with the flags: MT_DEVICE (like for stm32mp1) or
> > > MT_NON_CACHEABLE (for rpi3)
> > >
> > > So probably you're right. I will check with simple memcpy.
> >
> > There is a difference between MT_DEVICE and MT_NON_CACHEABLE: as far as
> > I know MT_DEVICE requires aligned accesses while MT_NON_CACHEABLE does
> > not.
> >
> > However, as I wrote in the other email, if I am not mistaken the current
> > implementation of memcpy might work well for us anyway. (To be
> > confirmed.)
>
> I am not entirely sure what exactly you want to confirm. I have already
> mentioned several time that our memcpy() on arm64 is using unaligned access.
> So it can't be used for copying data to/from device memory area.
I wrote it more clearly here:
https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2506221438250.8066@ubuntu-linux-20-04-desktop/
Assuming that the address passed to memcpy is 4K aligned, then it seems
to me that our memcpy implementation is using only aligned accesses.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-23 19:27 ` Stefano Stabellini
@ 2025-06-23 20:38 ` Julien Grall
2025-06-23 20:42 ` Stefano Stabellini
0 siblings, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-23 20:38 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Oleksii Moisieiev, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
Hi Stefano,
On 23/06/2025 20:27, Stefano Stabellini wrote:
> On Mon, 23 Jun 2025, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 22/06/2025 23:15, Stefano Stabellini wrote:
>>> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>>>> On 18/06/2025 02:22, Stefano Stabellini wrote:
>>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>>> [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
>>>>> Keep in mind that [0] refers specifically to access to MMIO regions. I
>>>>> assume that the SCMI shared buffers are on normal memory? Regarding [1],
>>>>> it makes sense if Linux is trying to support shared memory over MMIO.
>>>>>
>>>>> Looking at one of your replies below, I am guessing the memory buffers
>>>>> are actually in normal memory but the issue is that TF-A is mapping them
>>>>> as uncacheable. Is that correct?
>>>>>
>>>>> In that case, I still don't understand why a simple memcpy would not be
>>>>> sufficient. Can you check?
>>>>>
>>>>> If yes, then for now I would just simplify it down to memcpy. When
>>>>> someone adds support for an SCMI server elsewhere we could look into
>>>>> adding a more sophisticated memcpy and we can look at the details at
>>>>> that point in time. Specifically, I am not convinced that memcpy_toio
>>>>> and memcpy_fromio would work if the SCMI server is on a separate
>>>>> non-coherent microcontroller.
>>>>>
>>>> According to the TF-A implementation SCMI memory
>>>>
>>>> is mapped with the flags: MT_DEVICE (like for stm32mp1) or
>>>> MT_NON_CACHEABLE (for rpi3)
>>>>
>>>> So probably you're right. I will check with simple memcpy.
>>>
>>> There is a difference between MT_DEVICE and MT_NON_CACHEABLE: as far as
>>> I know MT_DEVICE requires aligned accesses while MT_NON_CACHEABLE does
>>> not.
>>>
>>> However, as I wrote in the other email, if I am not mistaken the current
>>> implementation of memcpy might work well for us anyway. (To be
>>> confirmed.)
>>
>> I am not entirely sure what exactly you want to confirm. I have already
>> mentioned several time that our memcpy() on arm64 is using unaligned access.
>> So it can't be used for copying data to/from device memory area.
>
> I wrote it more clearly here:
> https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2506221438250.8066@ubuntu-linux-20-04-desktop/
Ah I missed that e-mail!
>
> Assuming that the address passed to memcpy is 4K aligned, then it seems
> to me that our memcpy implementation is using only aligned accesses.
I didn't look at the mempcy() in details. But even if what you say is
true, it seems to be me this will be very fragile because we would assume:
* the addresses passed are always 4KB (I could not easily confirm it)
* the mempcy implementation will not change (I see Linux has updated
theirs in 2020 but we never did it...).
I can't think of a compiel time check that would help to confirm any
assumptions above will always hold true.
I also don't see what we would gain with implementing memcpy_toio() with
mempcy(). Maybe you can remind what's your concern with that?
So right now, I feel Oleksii approach is the best.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-23 20:38 ` Julien Grall
@ 2025-06-23 20:42 ` Stefano Stabellini
2025-06-23 21:58 ` Julien Grall
0 siblings, 1 reply; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-23 20:42 UTC (permalink / raw)
To: Julien Grall
Cc: Stefano Stabellini, Oleksii Moisieiev,
xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On Mon, 23 Jun 2025, Julien Grall wrote:
> Hi Stefano,
>
> On 23/06/2025 20:27, Stefano Stabellini wrote:
> > On Mon, 23 Jun 2025, Julien Grall wrote:
> > > Hi Stefano,
> > >
> > > On 22/06/2025 23:15, Stefano Stabellini wrote:
> > > > On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
> > > > > On 18/06/2025 02:22, Stefano Stabellini wrote:
> > > > > > On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
> > > > > > > [1]:https://git.iliana.fyi/linux/patch/?id=d5141f37c42e0b833863f157ac4cee203b2ba3d2
> > > > > > Keep in mind that [0] refers specifically to access to MMIO regions.
> > > > > > I
> > > > > > assume that the SCMI shared buffers are on normal memory? Regarding
> > > > > > [1],
> > > > > > it makes sense if Linux is trying to support shared memory over
> > > > > > MMIO.
> > > > > >
> > > > > > Looking at one of your replies below, I am guessing the memory
> > > > > > buffers
> > > > > > are actually in normal memory but the issue is that TF-A is mapping
> > > > > > them
> > > > > > as uncacheable. Is that correct?
> > > > > >
> > > > > > In that case, I still don't understand why a simple memcpy would not
> > > > > > be
> > > > > > sufficient. Can you check?
> > > > > >
> > > > > > If yes, then for now I would just simplify it down to memcpy. When
> > > > > > someone adds support for an SCMI server elsewhere we could look into
> > > > > > adding a more sophisticated memcpy and we can look at the details at
> > > > > > that point in time. Specifically, I am not convinced that
> > > > > > memcpy_toio
> > > > > > and memcpy_fromio would work if the SCMI server is on a separate
> > > > > > non-coherent microcontroller.
> > > > > >
> > > > > According to the TF-A implementation SCMI memory
> > > > >
> > > > > is mapped with the flags: MT_DEVICE (like for stm32mp1) or
> > > > > MT_NON_CACHEABLE (for rpi3)
> > > > >
> > > > > So probably you're right. I will check with simple memcpy.
> > > >
> > > > There is a difference between MT_DEVICE and MT_NON_CACHEABLE: as far as
> > > > I know MT_DEVICE requires aligned accesses while MT_NON_CACHEABLE does
> > > > not.
> > > >
> > > > However, as I wrote in the other email, if I am not mistaken the current
> > > > implementation of memcpy might work well for us anyway. (To be
> > > > confirmed.)
> > >
> > > I am not entirely sure what exactly you want to confirm. I have already
> > > mentioned several time that our memcpy() on arm64 is using unaligned
> > > access.
> > > So it can't be used for copying data to/from device memory area.
> >
> > I wrote it more clearly here:
> > https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2506221438250.8066@ubuntu-linux-20-04-desktop/
>
> Ah I missed that e-mail!
>
> >
> > Assuming that the address passed to memcpy is 4K aligned, then it seems
> > to me that our memcpy implementation is using only aligned accesses.
>
> I didn't look at the mempcy() in details. But even if what you say is true, it
> seems to be me this will be very fragile because we would assume:
> * the addresses passed are always 4KB (I could not easily confirm it)
> * the mempcy implementation will not change (I see Linux has updated theirs
> in 2020 but we never did it...).
>
> I can't think of a compiel time check that would help to confirm any
> assumptions above will always hold true.
>
> I also don't see what we would gain with implementing memcpy_toio() with
> mempcy(). Maybe you can remind what's your concern with that?
>
> So right now, I feel Oleksii approach is the best.
OK fair enough :-)
I was trying to avoid introducing two functions that seemed unnecessary.
If we go with Oleksii's approach, where do you think memcpy_toio()
should be added? Oleksii added them to the scmi file, maybe we want to
add them in a more generic location?
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-23 20:42 ` Stefano Stabellini
@ 2025-06-23 21:58 ` Julien Grall
0 siblings, 0 replies; 67+ messages in thread
From: Julien Grall @ 2025-06-23 21:58 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Oleksii Moisieiev, xen-devel@lists.xenproject.org, Andrew Cooper,
Anthony PERARD, Bertrand Marquis, Jan Beulich, Juergen Gross,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
Hi Stefano,
On 23/06/2025 21:42, Stefano Stabellini wrote:
> I was trying to avoid introducing two functions that seemed unnecessary.
> If we go with Oleksii's approach, where do you think memcpy_toio()
> should be added? Oleksii added them to the scmi file, maybe we want to
> add them in a more generic location?
As this is a generic implementation, I would consider to impkement it in
xen/lib/.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-22 21:30 ` Stefano Stabellini
@ 2025-06-24 8:42 ` Oleksii Moisieiev
2025-06-24 8:47 ` Jan Beulich
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-24 8:42 UTC (permalink / raw)
To: Stefano Stabellini, jbeulich@suse.com, roger.pau@citrix.com
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Volodymyr Babchuk, Grygorii Strashko, Roger Pau Monne,
Jan Beulich
Adding Roger and Jan to the conversation.
Please see below.
On 23/06/2025 00:30, Stefano Stabellini wrote:
> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>> Hi Stefano,
>>>>
>>>> I'm very sorry for a long silence. Please see my answers below:
>>>>
>>>> On 22/05/2025 03:25, Stefano Stabellini wrote:
>>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>>
>>>>>> Add chained handling of assigned DT devices to support access-controller
>>>>>> functionality through SCI framework, so DT device assign request can be
>>>>>> passed to FW for processing and enabling VM access to requested device
>>>>>> (for example, device power management through FW interface like SCMI).
>>>>>>
>>>>>> The SCI access-controller DT device processing is chained after IOMMU
>>>>>> processing and expected to be executed for any DT device regardless of its
>>>>>> protection by IOMMU (or if IOMMU is disabled).
>>>>>>
>>>>>> This allows to pass not only IOMMU protected DT device through
>>>>>> xl.cfg:"dtdev" property for processing:
>>>>>>
>>>>>> dtdev = [
>>>>>> "/soc/video@e6ef0000", <- IOMMU protected device
>>>>>> "/soc/i2c@e6508000", <- not IOMMU protected device
>>>>>> ]
>>>>>>
>>>>>> The change is done in two parts:
>>>>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>>>>>> not fail if DT device is not protected by IOMMU
>>>>>> 2) add chained call to sci_do_domctl() in do_domctl()
>>>>>>
>>>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>>>> ---
>>>>>>
>>>>>>
>>>>>>
>>>>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>>>>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>>>>>> xen/common/domctl.c | 19 +++++++++++++
>>>>>> xen/drivers/passthrough/device_tree.c | 6 ++++
>>>>>> 4 files changed, 76 insertions(+)
>>>>>>
>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>> +{
>>>>>> + struct dt_device_node *dev;
>>>>>> + int ret = 0;
>>>>>> +
>>>>>> + switch ( domctl->cmd )
>>>>>> + {
>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>> + ret = -EOPNOTSUPP;
>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>> XEN_DOMCTL_assign_device
>>>> call is expected to handle any DT device, regardless of whether the DT
>>>> device is
>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>> The following cases are considered:
>>>>
>>>> 1. IOMMU Protected Device (Success)
>>>>
>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>> we continue
>>>> processing the DT device by calling sci_do_domctl.
>>>>
>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>
>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>> disabled,
>>>> we still proceed to call sci_do_domctl.
>>> OK this makes sense. I think it is OK to have a special error code to
>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>> configuration with domctl disabled, for instance.
>>>
>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>
>> I see that in the following commit:
>>
>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>
>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>
>> It's not clear to me why this was done from the commit description.
>>
>> Maybe we should add commit author?
> Roger and Jan might know
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-24 8:42 ` Oleksii Moisieiev
@ 2025-06-24 8:47 ` Jan Beulich
2025-06-24 8:48 ` Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Jan Beulich @ 2025-06-24 8:47 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Volodymyr Babchuk, Grygorii Strashko, Stefano Stabellini,
roger.pau@citrix.com
On 24.06.2025 10:42, Oleksii Moisieiev wrote:
> Adding Roger and Jan to the conversation.
>
> Please see below.
Why is this? I did answer that question at the bottom already.
Jan
> On 23/06/2025 00:30, Stefano Stabellini wrote:
>> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>> Hi Stefano,
>>>>>
>>>>> I'm very sorry for a long silence. Please see my answers below:
>>>>>
>>>>> On 22/05/2025 03:25, Stefano Stabellini wrote:
>>>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>>>
>>>>>>> Add chained handling of assigned DT devices to support access-controller
>>>>>>> functionality through SCI framework, so DT device assign request can be
>>>>>>> passed to FW for processing and enabling VM access to requested device
>>>>>>> (for example, device power management through FW interface like SCMI).
>>>>>>>
>>>>>>> The SCI access-controller DT device processing is chained after IOMMU
>>>>>>> processing and expected to be executed for any DT device regardless of its
>>>>>>> protection by IOMMU (or if IOMMU is disabled).
>>>>>>>
>>>>>>> This allows to pass not only IOMMU protected DT device through
>>>>>>> xl.cfg:"dtdev" property for processing:
>>>>>>>
>>>>>>> dtdev = [
>>>>>>> "/soc/video@e6ef0000", <- IOMMU protected device
>>>>>>> "/soc/i2c@e6508000", <- not IOMMU protected device
>>>>>>> ]
>>>>>>>
>>>>>>> The change is done in two parts:
>>>>>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>>>>>>> not fail if DT device is not protected by IOMMU
>>>>>>> 2) add chained call to sci_do_domctl() in do_domctl()
>>>>>>>
>>>>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>>>>> ---
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>>>>>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>>>>>>> xen/common/domctl.c | 19 +++++++++++++
>>>>>>> xen/drivers/passthrough/device_tree.c | 6 ++++
>>>>>>> 4 files changed, 76 insertions(+)
>>>>>>>
>>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>>> return 0;
>>>>>>> }
>>>>>>>
>>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>>> +{
>>>>>>> + struct dt_device_node *dev;
>>>>>>> + int ret = 0;
>>>>>>> +
>>>>>>> + switch ( domctl->cmd )
>>>>>>> + {
>>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>>> + ret = -EOPNOTSUPP;
>>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>>> XEN_DOMCTL_assign_device
>>>>> call is expected to handle any DT device, regardless of whether the DT
>>>>> device is
>>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>>> The following cases are considered:
>>>>>
>>>>> 1. IOMMU Protected Device (Success)
>>>>>
>>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>>> we continue
>>>>> processing the DT device by calling sci_do_domctl.
>>>>>
>>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>>
>>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>>> disabled,
>>>>> we still proceed to call sci_do_domctl.
>>>> OK this makes sense. I think it is OK to have a special error code to
>>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>>> configuration with domctl disabled, for instance.
>>>>
>>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>>
>>> I see that in the following commit:
>>>
>>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>>
>>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>>
>>> It's not clear to me why this was done from the commit description.
>>>
>>> Maybe we should add commit author?
>> Roger and Jan might know
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-24 8:47 ` Jan Beulich
@ 2025-06-24 8:48 ` Oleksii Moisieiev
0 siblings, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-24 8:48 UTC (permalink / raw)
To: Jan Beulich
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Volodymyr Babchuk, Grygorii Strashko, Stefano Stabellini,
roger.pau@citrix.com
On 24/06/2025 11:47, Jan Beulich wrote:
> On 24.06.2025 10:42, Oleksii Moisieiev wrote:
>> Adding Roger and Jan to the conversation.
>>
>> Please see below.
> Why is this? I did answer that question at the bottom already.
>
> Jan
Oh sorry, missed this email..
>> On 23/06/2025 00:30, Stefano Stabellini wrote:
>>> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>>>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>>> Hi Stefano,
>>>>>>
>>>>>> I'm very sorry for a long silence. Please see my answers below:
>>>>>>
>>>>>> On 22/05/2025 03:25, Stefano Stabellini wrote:
>>>>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>>>>
>>>>>>>> Add chained handling of assigned DT devices to support access-controller
>>>>>>>> functionality through SCI framework, so DT device assign request can be
>>>>>>>> passed to FW for processing and enabling VM access to requested device
>>>>>>>> (for example, device power management through FW interface like SCMI).
>>>>>>>>
>>>>>>>> The SCI access-controller DT device processing is chained after IOMMU
>>>>>>>> processing and expected to be executed for any DT device regardless of its
>>>>>>>> protection by IOMMU (or if IOMMU is disabled).
>>>>>>>>
>>>>>>>> This allows to pass not only IOMMU protected DT device through
>>>>>>>> xl.cfg:"dtdev" property for processing:
>>>>>>>>
>>>>>>>> dtdev = [
>>>>>>>> "/soc/video@e6ef0000", <- IOMMU protected device
>>>>>>>> "/soc/i2c@e6508000", <- not IOMMU protected device
>>>>>>>> ]
>>>>>>>>
>>>>>>>> The change is done in two parts:
>>>>>>>> 1) update iommu_do_dt_domctl() to check for dt_device_is_protected() and
>>>>>>>> not fail if DT device is not protected by IOMMU
>>>>>>>> 2) add chained call to sci_do_domctl() in do_domctl()
>>>>>>>>
>>>>>>>> Signed-off-by: Grygorii Strashko<grygorii_strashko@epam.com>
>>>>>>>> Signed-off-by: Oleksii Moisieiev<oleksii_moisieiev@epam.com>
>>>>>>>> ---
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> xen/arch/arm/firmware/sci.c | 37 +++++++++++++++++++++++++
>>>>>>>> xen/arch/arm/include/asm/firmware/sci.h | 14 ++++++++++
>>>>>>>> xen/common/domctl.c | 19 +++++++++++++
>>>>>>>> xen/drivers/passthrough/device_tree.c | 6 ++++
>>>>>>>> 4 files changed, 76 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>>>> return 0;
>>>>>>>> }
>>>>>>>>
>>>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>>>> +{
>>>>>>>> + struct dt_device_node *dev;
>>>>>>>> + int ret = 0;
>>>>>>>> +
>>>>>>>> + switch ( domctl->cmd )
>>>>>>>> + {
>>>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>>>> + ret = -EOPNOTSUPP;
>>>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>>>> XEN_DOMCTL_assign_device
>>>>>> call is expected to handle any DT device, regardless of whether the DT
>>>>>> device is
>>>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>>>> The following cases are considered:
>>>>>>
>>>>>> 1. IOMMU Protected Device (Success)
>>>>>>
>>>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>>>> we continue
>>>>>> processing the DT device by calling sci_do_domctl.
>>>>>>
>>>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>>>
>>>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>>>> disabled,
>>>>>> we still proceed to call sci_do_domctl.
>>>>> OK this makes sense. I think it is OK to have a special error code to
>>>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>>>> configuration with domctl disabled, for instance.
>>>>>
>>>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>>>
>>>> I see that in the following commit:
>>>>
>>>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>>>
>>>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>>>
>>>> It's not clear to me why this was done from the commit description.
>>>>
>>>> Maybe we should add commit author?
>>> Roger and Jan might know
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver
2025-06-22 22:15 ` Stefano Stabellini
2025-06-23 8:06 ` Julien Grall
@ 2025-06-25 19:47 ` Oleksii Moisieiev
1 sibling, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-25 19:47 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Julien Grall,
Michal Orzel, Roger Pau Monné, Volodymyr Babchuk,
Grygorii Strashko
On 23/06/2025 01:15, Stefano Stabellini wrote:
> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>> On 18/06/2025 02:22, Stefano Stabellini wrote:
>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
[snip]
>>>>> Are you sure it is worth to go through all this trouble to modify FDT in
>>>>> place when we could simply generate the DT node from scratch like we do
>>>>> for example for the GIC? This seems to be more error prone as well. Is
>>>>> generating it from scratch is really difficult? If it is difficult then OK.
>>>>>
>>>> In the last patch [0] of the series, there is a proposal to separate
>>>>
>>>> the Xen privileged agent from the Dom0 agent. This eliminates the
>>>>
>>>> need to modify the Xen Device Tree Blob (DTB), which is a positive
>>>> improvement
>>>>
>>>> since the Dom0 agent node is already present in the device tree.
>>>>
>>>> However, the issue with SCMI node generation lies in the need to handle
>>>>
>>>> the list of protocols that are supported across the system.
>>>>
>>>> If we want to generate the SCMI node for Dom0, we need to:
>>>>
>>>> Copy these protocols from a centralized or predefined source.
>>>> Set the correct phandle for each subnode within the SCMI node, ensuring
>>>>
>>>> accurate representation and functionality.
>>>> This extra step of managing the protocols and phandles adds
>>>>
>>>> complexity but is necessary for ensuring proper support for Dom0.
>>>>
>>>> [0]
>>>> https://lists.xenproject.org/archives/html/xen-devel/2025-05/msg01041.html
>>> I was only commenting that rather than trying to modify the DT in place
>>> we could create the node for Dom0 from scratch (artificially), based on
>>> host DT information as required (fetching data from the host DT as
>>> required and copying it to the Dom0 DT).
>>>
>> The problem is that scmi node, apart from the default parameters has a
>> list of
>>
>> supported protocols which may be different for Xen and Dom0 so to
>> generate Dom0 node
>>
>> from scratch we need to copy(or even generate) these nodes which will
>> require complex implementation.
>>
>> In contrary, if we go with implementation, proposed in the last patch -
>> then we can have
>>
>> more cleaner Xen device tree, which will look like this:
>>
>> ```
>>
>> chosen {
>>
>> scmi_xen: scmi {
>>
>> arm,smc-id = <0x82000002>;
>>
>> shmem= <$shm_0>;
>>
>> ... # no protocols description here
>>
>> };
>>
>> firmware {
>>
>> scmi {
>>
>> arm,smc-id = <0x82000003>;
>>
>> shmem= <$shm_1>;
>>
>> protocol@X{
>>
>> };
>>
>> }
>>
>> };
>>
>> ```
>>
>> In this case, we don’t provide any protocol information for the Xen
>> agent and instead set protocols for Dom0.
>>
>> This ensures that each node has the information it requires.
> Leaving aside how we generate the device tree nodes for Dom0, which is
> of minor importance, we have two potentially conflicting goals:
>
> 1) no Xen-specific changes to Device Tree outside of /chosen
> 2) When Xen is present, Xen should be the one using the information in the Host Device Tree
>
> With 1), I mean that the Host Device Tree, outside of the /chosen node,
> should be the same for the BSP (no Xen) and for Xen. That is because it
> should describe the platform, no matter the software running on it. Also
> at the time the Device Tree is generated, often it is not known if Xen
> will be running on the platform or not.
>
> With 2) I meant that ideally when Xen is present, Xen should be the one
> to use the information under /firmware/scmi, because Xen is the
> principal SCMI client. Then Xen would generate somehow the SCMI device
> tree nodes for Dom0 and the DomUs.
>
> 1) is more important than 2).
>
> >From what you wrote, it looks like the SCMI device tree nodes are
> different between Linux baremetal and Xen. If /firmware/scmi describes
> what Linux baremetal should use, then it would work for Linux baremetal
> but it would *not* work for Xen's own SCMI connection. Is that correct?
>
> If that is the case, that is unfortunate. We need to compromise on 2).
>
> I would keep /firmware/scmi as the one used by Linux baremetal because I
> think we should uphold 1) above all else.
>
> Then the Xen SCMI nodes could be placed under /chosen. This is less than
> ideal because /chosen should contain Xen/Linux configurations, not
> firmware interfaces. But it is the best we can do in a difficult
> situation.
>
> I think that whether the Dom0 device tree nodes are artificially
> generated by Xen, or copied from the Host Device Tree somewhere, it
> matters less.
I completely agree with your point. That’s why, during the discussion of
the last patch in the series, I proposed an approach
that keeps the /firmware/scmi node unchanged in the Host Device Tree
(DT) and places all Xen-related
changes inside the /chosen/xen,config node.
I plan to follow this approach in v5.
Here’s the link to the message where I provided a DT example that
ensures the /firmware/scmi node remains the same as in the Host DT: [0].
[0]:
https://lists.xenproject.org/archives/html/xen-devel/2025-06/msg01421.html
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-23 8:02 ` Julien Grall
@ 2025-06-25 19:47 ` Oleksii Moisieiev
2025-06-25 20:32 ` Julien Grall
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-25 19:47 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On 23/06/2025 11:02, Julien Grall wrote:
> Hi Stefano and Oleksii,
>
> Let me start with a bit of process. This is discussion is getting
> fairly difficult to follow.... Can you please trim unrelevant bits
> when replying?
>
> On 22/06/2025 22:57, Stefano Stabellini wrote:
>> On Thu, 19 Jun 2025, Oleksii Moisieiev wrote:
>>> On 18/06/2025 03:35, Stefano Stabellini wrote:
>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>> On 23/05/2025 23:19, Stefano Stabellini wrote:
>>>>>> On Mon, 19 May 2025, Oleksii Moisieiev wrote:
>>>>>>> From: Grygorii Strashko<grygorii_strashko@epam.com>
>>> the same (smc-id and shmem) for both the BSP case (no Xen) and the Xen
>>> case (Dom0 domain).
>>>
>>> Meanwhile, the Xen management agent's SCMI node and configuration are
>>> expected to be placed under /chosen.
>>>
>>> This approach ensures that the Host DT remains as unchanged as
>>> possible.
>>
>> Yes, my main point is that all the device tree information, except for
>> what is under /chosen, should be left unchanged between the BSP case (no
>> Xen) and the Xen case.
>>
>> We have freedom to decide:
>> - the information we put under /chosen and how to interpret it
>> - how to use the information under /firmware/scmi when Xen is present
>>
>>
>>> Currently:
>>>
>>> The Host DT /firmware/scmi node requires modification to point to the
>>> Xen management agent by changing
>>>
>>> the smc-id and shmem values.
>>
>> I don't think we should require changes to /firmware/scmi in the host DT
>> when Xen is present.
>>
>> Often, people don't know when or if Xen is present at the time the
>> Device Tree is generated. So it is best to avoid modification (outside
>> of /chosen).
>
> I am probably missing something. But it looks like TF-A requires to
> suport multi-agent so Xen can use it. Am I correct?
>
> Furthermore, I can't tell why the multi-agent support is Xen specific.
> Surely, you may want something similar with other hypervisors? If not,
> then my next question is why does Xen needs to do things differently?
>
> Cheers,
>
Yes, multi-agent support is required in TF-A for Xen, but this is not
specific to Xen.
The implementation is based on the scenario described in the last
paragraph of
section 4.2.1 of DEN0056 [0].
The key points are as follows:
- a Virtual Machine (VM) serves as a non-trusted agent in the
Non-secure Security state.
- a hypervisor acts as a trusted agent in the Non-secure
Security state.
- the hypervisor can configure fine-grained Non-secure resource access
permissions for
Virtual Machines.
- the hypervisor sets resource access permissions for the agent
identifier associated with
the channel and assigns the channel to the VM.
Therefore, we can expect other hypervisors to follow this specification.
[0]: https://developer.arm.com/documentation/den0056/latest/
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-23 7:15 ` Jan Beulich
2025-06-23 7:28 ` Oleksii Moisieiev
@ 2025-06-25 19:56 ` Oleksii Moisieiev
2025-06-26 6:10 ` Jan Beulich
1 sibling, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-25 19:56 UTC (permalink / raw)
To: Jan Beulich
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko,
Stefano Stabellini
On 23/06/2025 10:15, Jan Beulich wrote:
> On 19.06.2025 18:15, Oleksii Moisieiev wrote:
>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>> +{
>>>>>> + struct dt_device_node *dev;
>>>>>> + int ret = 0;
>>>>>> +
>>>>>> + switch ( domctl->cmd )
>>>>>> + {
>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>> + ret = -EOPNOTSUPP;
>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>> XEN_DOMCTL_assign_device
>>>> call is expected to handle any DT device, regardless of whether the DT
>>>> device is
>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>> The following cases are considered:
>>>>
>>>> 1. IOMMU Protected Device (Success)
>>>>
>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>> we continue
>>>> processing the DT device by calling sci_do_domctl.
>>>>
>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>
>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>> disabled,
>>>> we still proceed to call sci_do_domctl.
>>> OK this makes sense. I think it is OK to have a special error code to
>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>> configuration with domctl disabled, for instance.
>>>
>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>
>> I see that in the following commit:
>>
>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>
>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>
>> It's not clear to me why this was done from the commit description.
> This has been discussed many times elsewhere. Many of our ENOSYS uses are
> simply wrong. ENOSYS has very limited applicability: Unavailability of a
> top-level hypercall (originally: syscall).
>
What is your opinion about changing it to -ENOENT to say "the IOMMU is
disabled" as Stefano suggested in [0]?
[0]: https://lists.xen.org/archives/html/xen-devel/2025-06/msg01233.html
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-25 19:47 ` Oleksii Moisieiev
@ 2025-06-25 20:32 ` Julien Grall
2025-06-29 15:41 ` Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-25 20:32 UTC (permalink / raw)
To: Oleksii Moisieiev, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
Hi Oleksii,
On 25/06/2025 20:47, Oleksii Moisieiev wrote:
>
> On 23/06/2025 11:02, Julien Grall wrote:
>> I am probably missing something. But it looks like TF-A requires to
>> suport multi-agent so Xen can use it. Am I correct?
>>
>> Furthermore, I can't tell why the multi-agent support is Xen specific.
>> Surely, you may want something similar with other hypervisors? If not,
>> then my next question is why does Xen needs to do things differently?
>>
>> Cheers,
>>
>
> Yes, multi-agent support is required in TF-A for Xen, but this is not
> specific to Xen.
I am really confused. If the support is not Xen specific then why do we
end up to have xen specific node/properties in your proposal (see [1])
such as xen,scmi-secondary-agents.
I also question the placement of the SMCI multi-agent in /chosen. For me
/chosen is for configuration related to the hypervisor/OS. But here, it
seems the multi-agent SMCI is related to the platform.
So wouldn't it be better to create a new compatible arm,smci-multi that
will include the information for multi? An alternative would be to
extend the existing SCMI node in a backward compatible way.
Lastly, I see if you put a node under "/chosen" with "arm,scmi-smc".
Have you checked this will not confused Linux? I was under the
impression Linux would look for any node with the compatible when
initializing a driver.
Cheers,
[1]
https://lists.xenproject.org/archives/html/xen-devel/2025-06/msg01421.html
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-25 19:56 ` Oleksii Moisieiev
@ 2025-06-26 6:10 ` Jan Beulich
2025-06-26 13:07 ` Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Jan Beulich @ 2025-06-26 6:10 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko,
Stefano Stabellini
On 25.06.2025 21:56, Oleksii Moisieiev wrote:
>
> On 23/06/2025 10:15, Jan Beulich wrote:
>> On 19.06.2025 18:15, Oleksii Moisieiev wrote:
>>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>>> return 0;
>>>>>>> }
>>>>>>>
>>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>>> +{
>>>>>>> + struct dt_device_node *dev;
>>>>>>> + int ret = 0;
>>>>>>> +
>>>>>>> + switch ( domctl->cmd )
>>>>>>> + {
>>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>>> + ret = -EOPNOTSUPP;
>>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>>> XEN_DOMCTL_assign_device
>>>>> call is expected to handle any DT device, regardless of whether the DT
>>>>> device is
>>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>>> The following cases are considered:
>>>>>
>>>>> 1. IOMMU Protected Device (Success)
>>>>>
>>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>>> we continue
>>>>> processing the DT device by calling sci_do_domctl.
>>>>>
>>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>>
>>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>>> disabled,
>>>>> we still proceed to call sci_do_domctl.
>>>> OK this makes sense. I think it is OK to have a special error code to
>>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>>> configuration with domctl disabled, for instance.
>>>>
>>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>>
>>> I see that in the following commit:
>>>
>>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>>
>>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>>
>>> It's not clear to me why this was done from the commit description.
>> This has been discussed many times elsewhere. Many of our ENOSYS uses are
>> simply wrong. ENOSYS has very limited applicability: Unavailability of a
>> top-level hypercall (originally: syscall).
>>
> What is your opinion about changing it to -ENOENT to say "the IOMMU is
> disabled" as Stefano suggested in [0]?
>
> [0]: https://lists.xen.org/archives/html/xen-devel/2025-06/msg01233.html
To me, ENOENT is closer to ENODEV, and hence not overly applicable here.
If you want to avoid EOPNOTSUPP for whatever reason, how about ENXIO or
EIO? (EPERM might also be an option, but we assign that a different
meaning generally.)
Jan
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-26 6:10 ` Jan Beulich
@ 2025-06-26 13:07 ` Oleksii Moisieiev
2025-06-26 14:41 ` Jan Beulich
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-26 13:07 UTC (permalink / raw)
To: Jan Beulich
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko,
Stefano Stabellini
On 26/06/2025 09:10, Jan Beulich wrote:
> On 25.06.2025 21:56, Oleksii Moisieiev wrote:
>> On 23/06/2025 10:15, Jan Beulich wrote:
>>> On 19.06.2025 18:15, Oleksii Moisieiev wrote:
>>>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>>>> return 0;
>>>>>>>> }
>>>>>>>>
>>>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>>>> +{
>>>>>>>> + struct dt_device_node *dev;
>>>>>>>> + int ret = 0;
>>>>>>>> +
>>>>>>>> + switch ( domctl->cmd )
>>>>>>>> + {
>>>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>>>> + ret = -EOPNOTSUPP;
>>>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>>>> XEN_DOMCTL_assign_device
>>>>>> call is expected to handle any DT device, regardless of whether the DT
>>>>>> device is
>>>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>>>> The following cases are considered:
>>>>>>
>>>>>> 1. IOMMU Protected Device (Success)
>>>>>>
>>>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>>>> we continue
>>>>>> processing the DT device by calling sci_do_domctl.
>>>>>>
>>>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>>>
>>>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>>>> disabled,
>>>>>> we still proceed to call sci_do_domctl.
>>>>> OK this makes sense. I think it is OK to have a special error code to
>>>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>>>> configuration with domctl disabled, for instance.
>>>>>
>>>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>>>
>>>> I see that in the following commit:
>>>>
>>>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>>>
>>>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>>>
>>>> It's not clear to me why this was done from the commit description.
>>> This has been discussed many times elsewhere. Many of our ENOSYS uses are
>>> simply wrong. ENOSYS has very limited applicability: Unavailability of a
>>> top-level hypercall (originally: syscall).
>>>
>> What is your opinion about changing it to -ENOENT to say "the IOMMU is
>> disabled" as Stefano suggested in [0]?
>>
>> [0]: https://lists.xen.org/archives/html/xen-devel/2025-06/msg01233.html
> To me, ENOENT is closer to ENODEV, and hence not overly applicable here.
> If you want to avoid EOPNOTSUPP for whatever reason, how about ENXIO or
> EIO? (EPERM might also be an option, but we assign that a different
> meaning generally.)
>
> Jan
Maybe -ENODEV is a better choice because iommu_do_pci_domctl and
iommu_do_dt_domctl return this
code when some feature is not supported.
I think -EIO or -ENXIO aren’t suitable here since we’re planning to send
this message when the IOMMU is disabled.
What do you think?
WBR,
Oleksii.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-26 13:07 ` Oleksii Moisieiev
@ 2025-06-26 14:41 ` Jan Beulich
2025-06-26 15:01 ` Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Jan Beulich @ 2025-06-26 14:41 UTC (permalink / raw)
To: Oleksii Moisieiev
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko,
Stefano Stabellini
On 26.06.2025 15:07, Oleksii Moisieiev wrote:
>
> On 26/06/2025 09:10, Jan Beulich wrote:
>> On 25.06.2025 21:56, Oleksii Moisieiev wrote:
>>> On 23/06/2025 10:15, Jan Beulich wrote:
>>>> On 19.06.2025 18:15, Oleksii Moisieiev wrote:
>>>>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>>>>> return 0;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>>>>> +{
>>>>>>>>> + struct dt_device_node *dev;
>>>>>>>>> + int ret = 0;
>>>>>>>>> +
>>>>>>>>> + switch ( domctl->cmd )
>>>>>>>>> + {
>>>>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>>>>> + ret = -EOPNOTSUPP;
>>>>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>>>>> XEN_DOMCTL_assign_device
>>>>>>> call is expected to handle any DT device, regardless of whether the DT
>>>>>>> device is
>>>>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>>>>> The following cases are considered:
>>>>>>>
>>>>>>> 1. IOMMU Protected Device (Success)
>>>>>>>
>>>>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>>>>> we continue
>>>>>>> processing the DT device by calling sci_do_domctl.
>>>>>>>
>>>>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>>>>
>>>>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>>>>> disabled,
>>>>>>> we still proceed to call sci_do_domctl.
>>>>>> OK this makes sense. I think it is OK to have a special error code to
>>>>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>>>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>>>>> configuration with domctl disabled, for instance.
>>>>>>
>>>>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>>>>
>>>>> I see that in the following commit:
>>>>>
>>>>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>>>>
>>>>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>>>>
>>>>> It's not clear to me why this was done from the commit description.
>>>> This has been discussed many times elsewhere. Many of our ENOSYS uses are
>>>> simply wrong. ENOSYS has very limited applicability: Unavailability of a
>>>> top-level hypercall (originally: syscall).
>>>>
>>> What is your opinion about changing it to -ENOENT to say "the IOMMU is
>>> disabled" as Stefano suggested in [0]?
>>>
>>> [0]: https://lists.xen.org/archives/html/xen-devel/2025-06/msg01233.html
>> To me, ENOENT is closer to ENODEV, and hence not overly applicable here.
>> If you want to avoid EOPNOTSUPP for whatever reason, how about ENXIO or
>> EIO? (EPERM might also be an option, but we assign that a different
>> meaning generally.)
>
> Maybe -ENODEV is a better choice because iommu_do_pci_domctl and
> iommu_do_dt_domctl return this
>
> code when some feature is not supported.
What feature are you talking about? All I see in the former is
ret = -ENODEV;
if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_PCI )
break;
and there -ENODEV is quite appropriate.
> I think -EIO or -ENXIO aren’t suitable here since we’re planning to send
> this message when the IOMMU is disabled.
Well, I don't like those two very much either for the use here, but I still
view them as better than ENOENT.
Jan
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu
2025-06-26 14:41 ` Jan Beulich
@ 2025-06-26 15:01 ` Oleksii Moisieiev
0 siblings, 0 replies; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-26 15:01 UTC (permalink / raw)
To: Jan Beulich
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Juergen Gross, Julien Grall, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko,
Stefano Stabellini
On 26/06/2025 17:41, Jan Beulich wrote:
> On 26.06.2025 15:07, Oleksii Moisieiev wrote:
>> On 26/06/2025 09:10, Jan Beulich wrote:
>>> On 25.06.2025 21:56, Oleksii Moisieiev wrote:
>>>> On 23/06/2025 10:15, Jan Beulich wrote:
>>>>> On 19.06.2025 18:15, Oleksii Moisieiev wrote:
>>>>>> On 18/06/2025 03:04, Stefano Stabellini wrote:
>>>>>>> On Thu, 12 Jun 2025, Oleksii Moisieiev wrote:
>>>>>>>>>> diff --git a/xen/arch/arm/firmware/sci.c b/xen/arch/arm/firmware/sci.c
>>>>>>>>>> index e1522e10e2..8efd541c4f 100644
>>>>>>>>>> --- a/xen/arch/arm/firmware/sci.c
>>>>>>>>>> +++ b/xen/arch/arm/firmware/sci.c
>>>>>>>>>> @@ -126,6 +126,43 @@ int sci_assign_dt_device(struct domain *d, struct dt_device_node *dev)
>>>>>>>>>> return 0;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> +int sci_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>>>>>>>>> + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>>>>>>>>>> +{
>>>>>>>>>> + struct dt_device_node *dev;
>>>>>>>>>> + int ret = 0;
>>>>>>>>>> +
>>>>>>>>>> + switch ( domctl->cmd )
>>>>>>>>>> + {
>>>>>>>>>> + case XEN_DOMCTL_assign_device:
>>>>>>>>>> + ret = -EOPNOTSUPP;
>>>>>>>>> Are you sure -EOPNOTSUPP is the right error code for the 3 checks below?
>>>>>>>> The -EOPNOTSUPP code is used because this is part of a chained call after
>>>>>>>> iommu_do_domctl, as stated in xen/common/domctl.c:859. The
>>>>>>>> XEN_DOMCTL_assign_device
>>>>>>>> call is expected to handle any DT device, regardless of whether the DT
>>>>>>>> device is
>>>>>>>> protected by an IOMMU or if the IOMMU is disabled.
>>>>>>>> The following cases are considered:
>>>>>>>>
>>>>>>>> 1. IOMMU Protected Device (Success)
>>>>>>>>
>>>>>>>> If the device is protected by the IOMMU and iommu_do_domctl returns 0,
>>>>>>>> we continue
>>>>>>>> processing the DT device by calling sci_do_domctl.
>>>>>>>>
>>>>>>>> 2. IOMMU Disabled (-EOPNOTSUPP from iommu_do_domctl)
>>>>>>>>
>>>>>>>> If iommu_do_domctl returns -EOPNOTSUPP, indicating that the IOMMU is
>>>>>>>> disabled,
>>>>>>>> we still proceed to call sci_do_domctl.
>>>>>>> OK this makes sense. I think it is OK to have a special error code to
>>>>>>> say "the IOMMU is disabled" but I don't know if it is a good idea to try
>>>>>>> to use -EOPNOTSUPP for that. -EOPNOTSUPP could mean a hypervisor
>>>>>>> configuration with domctl disabled, for instance.
>>>>>>>
>>>>>>> It might be wiser to use a different error code. Maybe ENOENT?
>>>>>>>
>>>>>> I see that in the following commit:
>>>>>>
>>>>>> 71e617a6b8 (use is_iommu_enabled() where appropriate..., 2019-09-17)
>>>>>>
>>>>>> -ENOSYS return code was changed to -EOPNOTSUPP in iommu_do_domctl.
>>>>>>
>>>>>> It's not clear to me why this was done from the commit description.
>>>>> This has been discussed many times elsewhere. Many of our ENOSYS uses are
>>>>> simply wrong. ENOSYS has very limited applicability: Unavailability of a
>>>>> top-level hypercall (originally: syscall).
>>>>>
>>>> What is your opinion about changing it to -ENOENT to say "the IOMMU is
>>>> disabled" as Stefano suggested in [0]?
>>>>
>>>> [0]: https://lists.xen.org/archives/html/xen-devel/2025-06/msg01233.html
>>> To me, ENOENT is closer to ENODEV, and hence not overly applicable here.
>>> If you want to avoid EOPNOTSUPP for whatever reason, how about ENXIO or
>>> EIO? (EPERM might also be an option, but we assign that a different
>>> meaning generally.)
>> Maybe -ENODEV is a better choice because iommu_do_pci_domctl and
>> iommu_do_dt_domctl return this
>>
>> code when some feature is not supported.
> What feature are you talking about? All I see in the former is
>
> ret = -ENODEV;
> if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_PCI )
> break;
>
> and there -ENODEV is quite appropriate.
I was talking about the following code in iommu_do_pci_domctl:
ret = -ENODEV;
if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_PCI )
break;
Sorry, I misinterpreted this.
>> I think -EIO or -ENXIO aren’t suitable here since we’re planning to send
>> this message when the IOMMU is disabled.
> Well, I don't like those two very much either for the use here, but I still
> view them as better than ENOENT.
>
I think I could use -ENXIO since it has the following comment:
/* No such device or address */
It seems to be suitable for our case.
WBR,
Oleksii.
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-25 20:32 ` Julien Grall
@ 2025-06-29 15:41 ` Oleksii Moisieiev
2025-06-29 18:34 ` Julien Grall
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-29 15:41 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On 25/06/2025 23:32, Julien Grall wrote:
> Hi Oleksii,
>
> On 25/06/2025 20:47, Oleksii Moisieiev wrote:
>>
>> On 23/06/2025 11:02, Julien Grall wrote:
>>> I am probably missing something. But it looks like TF-A requires to
>>> suport multi-agent so Xen can use it. Am I correct?
>>>
>>> Furthermore, I can't tell why the multi-agent support is Xen specific.
>>> Surely, you may want something similar with other hypervisors? If not,
>>> then my next question is why does Xen needs to do things differently?
>>>
>>> Cheers,
>>>
>>
>> Yes, multi-agent support is required in TF-A for Xen, but this is not
>> specific to Xen.
>
> I am really confused. If the support is not Xen specific then why do
> we end up to have xen specific node/properties in your proposal (see
> [1]) such as xen,scmi-secondary-agents.
>
Sorry, for confusion. Some additional explanation is below.
All this is about the case when Host platform doesn't have SCP and SCMI
implemented in EL2 Firmware (TF-A).
The SCMI standard defines multi-agent support (as many other features),
but it is Optional.
Also SCMI standard doesn't define how its features has to be implemented
internally, just defines standard interface (API/HAL) to access them.
More over, even "shared-ram" Transport implementation is not strictly
standardized, for example in terms of
"shared-ram" base address alignment.
In most of the cases, SoC Vendors (if we are lucky) provide BSP with EL2
SCMI FW which implements
one SCMI transport and, so supports only one Agent which, in general,
has access to all resources.
Such EL2 SCMI FW is NOT virtualization (Xen) aware, so only "Simple SCMI
over SMC/HVC calls forwarding driver (EL3)" (CONFIG_SCMI_SMC)
can be used out of the box to enable SCMI in Hardware domain (or, with
patches 1-3 of this series, in Driver domain).
So, to have SCMI multi-channel features enabled with EL2 SCMI FW - it
has some SCMI features to be implemented
(which otherwise optional), including defining additional Transport
channels (Agents), shmem alignment on Xen page boundary,..
The requirements for EL2 SCMI FW is described in SCMI documentation [2].
And generic answer on your question above "I am probably missing
something. But it looks like TF-A requires to
suport multi-agent so Xen can use it. Am I correct?" would be Yes.
[1]
https://github.com/oleksiimoisieiev/xen/blob/scmi_upstrv5/docs/hypervisor-guide/arm/firmware/arm-scmi.rst#simple-scmi-over-smchvc-calls-forwarding-driver-el3
[2]
https://github.com/oleksiimoisieiev/xen/blob/scmi_upstrv5/docs/hypervisor-guide/arm/firmware/arm-scmi.rst#scmi-smchvc-multi-agent-driver-el3
-----
SCMI SCP Note. When Host platform support SCP the multi-agent support
will be in place, as
in such cases separate SCMI transport is allocated for at least each
Application processor (AP) which is defined as SCMI Agent.
For example, Core-R(s) (like Cortex-R/M), DSPs, Core-A (Cortex-A - here
the Xen is usually running).
No Agents in the system knows about each other. Only SCP FW does.
And even in this case, the BSP SCP FW might NOT be virtualization (Xen)
aware as only one Transport (Agent)
is provided for Core-A (actually two - one Secure for PSCI, and one
non-secure - for OSPM).
---
In the Virtualized system:
- The Xen is real OSPM which manages other Virtual OSPM and it needs
dedicated SCMI management channel through which
it can perform HW resources assignment for Virtual OSPM by
communicating with EL2 SCMI FW
during Virtual OSPM creation or release HW resources during Virtual
OSPM destruction.
In the future it also possible to enable such PM feature as SCMI based
CpuFreq in Xen.
The SCMI DT binding for Xen SCMI Agent expected to be exactly the same
as for any OSPM (like Linux) as from
SCMI FW point of few it is just OS running on Application Core (which
substitutes regular OS - Linux, RTOS).
So no new SCMI specific bindings (including compatibilities)
introduced neither in this series nor in this proposal.
In other words, Xen needs two DT to boot, kinda:
- Xen DT (with bootinfo, Application Core info, uart)
- Host Platform DT (source information to create domains)
and if there would be two separate DTs - each will have own standard
(bindings) DT SCMI node.
According to the current design Xen accepts DT which is Host Platform DT
with added Xen configuration so Xen SCMI info is added in Xen
configuration node under /chosen, and no Domains is expected to see it
ever. After Xen initialization DT nodes from Xen DT are copied to the
Dom0 device tree. Our proposal is to keep SCMI configuration from
Platform Host DT the same so it will be copied to the Dom0 device tree.
- the number of Virtual Machines or Virtual OSPMs (in terms of SCMI)
depends on hypervisor (Xen) configuration.
And Virtual OSPM is defined as VM which has direct access to HW
(passthrough), so need access
SCMI services to get fine-grained and safe access to required Platform
HW resources, and avoid interference.
Every Virtual OSPM is SCMI agent, which sees it's own SCMI transport,
and doesn't know about other agents.
In the case of DT - the standard SCMI bindings are used.
- So the Xen is the only entity in the platform which need to know about
other Agents.
Therefore, this Xen specific configuration "xen,scmi-secondary-agents",
for the case of the EL2 SCMI FW, is introduced and added under the
/chosen node (or xen-config).
Unfortunately, there is no way to discover Agent's configurations
using SCMI protocol (base), like "func-id"
and shmem parameter (only can get Number of Agents, and discover own
Agent id), so only option is to
define this info in DT for Xen. However, Xen can use shared memory
regions and func_ids of the other Agents to determine agent_id using
base protocol. That's why it was decided to make agent_id in
"xem,scmi-secondary-agents" optional.
> I also question the placement of the SMCI multi-agent in /chosen. For
> me /chosen is for configuration related to the hypervisor/OS. But
> here, it seems the multi-agent SMCI is related to the platform.
>
From SCMI point of view the Xen is separate OSPM (SCMI Agent), which
have own boot configuration data, in case of ARM - Device-tree.
The SCMI doesn't see the difference between Linux or Xen running on
Application processor.
They are both OSPM (Agents), but with different privileges and permissions.
As was stated before from Xen standpoint the device tree should be same
as Host DT with Xen
configuration placed under /chosen node. So agent_id dedicated to Xen
should be in xen configuration node.
The Host platform (BSP) will run using Host DT with Xen DT data removed and
doesn't need to know about Xen SCMI management channel (agent) or agents
available for Virtual OSPMs (VMs).
For Xen to run with SCMI multi-channel support - Xen specific DT data has
to be added under /chosen which includes Xen specific SCMI data.
> So wouldn't it be better to create a new compatible arm,smci-multi
> that will include the information for multi? An alternative would be
> to extend the existing SCMI node in a backward compatible way.
>
> Lastly, I see if you put a node under "/chosen" with "arm,scmi-smc".
> Have you checked this will not confused Linux? I was under the
> impression Linux would look for any node with the compatible when
> initializing a driver.
>
Xen configuration under /chosen will be removed when passing DT to the
Domains.
So Linux domains will not see any Xen configuration data.
When booting Baremetal Xen DT Linux kernel will not parse "arm,scmi-smc"
node under "chosen" as Linux uses 2 depth level for probing.
And there is no compatible in chosen:
{
chosen {
// no compatible - skip
xen-config {
compat = "xen,config"; <- no go below as not Bus compat
scmi {
<-never get here
}
}
}
}
> Cheers,
>
> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2025-06/msg01421.html
>
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-29 15:41 ` Oleksii Moisieiev
@ 2025-06-29 18:34 ` Julien Grall
2025-06-30 11:57 ` Oleksii Moisieiev
0 siblings, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-29 18:34 UTC (permalink / raw)
To: Oleksii Moisieiev, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
Hi Oleksii,
On 29/06/2025 16:41, Oleksii Moisieiev wrote:
> --->
> In the Virtualized system:
Thanks for the long explanation. In this section, you mention Xen all
over the place. But as you previously agreed the multi-agent SCMI is not
Xen specific. To put it differently, aside the fact...
>
> - The Xen is real OSPM which manages other Virtual OSPM and it needs>
dedicated SCMI management channel through which
> it can perform HW resources assignment for Virtual OSPM by
> communicating with EL2 SCMI FW
> during Virtual OSPM creation or release HW resources during Virtual
> OSPM destruction.
> In the future it also possible to enable such PM feature as SCMI based
> CpuFreq in Xen.
>
> The SCMI DT binding for Xen SCMI Agent expected to be exactly the same
> as for any OSPM (like Linux) as from
> SCMI FW point of few it is just OS running on Application Core (which
> substitutes regular OS - Linux, RTOS).
> So no new SCMI specific bindings (including compatibilities)
> introduced neither in this series nor in this proposal.
>
> In other words, Xen needs two DT to boot, kinda:
> - Xen DT (with bootinfo, Application Core info, uart)
> - Host Platform DT (source information to create domains)
> and if there would be two separate DTs - each will have own standard
> (bindings) DT SCMI node.
> According to the current design Xen accepts DT which is Host Platform DT
> with added Xen configuration so Xen SCMI info is added in Xen
> configuration node under /chosen, and no Domains is expected to see it
> ever. After Xen initialization DT nodes from Xen DT are copied to the
> Dom0 device tree. Our proposal is to keep SCMI configuration from
> Platform Host DT the same so it will be copied to the Dom0 device tree.
>
>
> - the number of Virtual Machines or Virtual OSPMs (in terms of SCMI)
> depends on hypervisor (Xen) configuration.
> And Virtual OSPM is defined as VM which has direct access to HW
> (passthrough), so need access
> SCMI services to get fine-grained and safe access to required Platform
> HW resources, and avoid interference.
>
> Every Virtual OSPM is SCMI agent, which sees it's own SCMI transport,
> and doesn't know about other agents.
> In the case of DT - the standard SCMI bindings are used.
>
> - So the Xen is the only entity in the platform which need to know about
> other Agents.
> Therefore, this Xen specific configuration "xen,scmi-secondary-agents",
> for the case of the EL2 SCMI FW, is introduced and added under the
> /chosen node (or xen-config).
> Unfortunately, there is no way to discover Agent's configurations
> using SCMI protocol (base), like "func-id"
> and shmem parameter (only can get Number of Agents, and discover own
> Agent id), so only option is to
> define this info in DT for Xen. However, Xen can use shared memory
> regions and func_ids of the other Agents to determine agent_id using
> base protocol. That's why it was decided to make agent_id in
> "xem,scmi-secondary-agents" optional.
... the name of this property contains "xen", I still don't understand
why the binding could not be used by other hypervisors. IOW, what if
above you s/xen/KVM/ (or any other hypervisor you come up with)? Are
they all going to create their own bindings? I would guess not given the
single agent is already generic (if I am not mistaken, both Linux and
Xen are using it)
I will not insist on moving the binding outside of /chosen if the other
maintainers think this is the best. But I think this is shortsighted to
add "xen" in all the name or put it in a Xen specific position.
Ultimately what I want to avoid is we have to support multiple bindings
in Xen because someone else decided to create a new binding as we didn't
even attempt to make ours generic...
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-29 18:34 ` Julien Grall
@ 2025-06-30 11:57 ` Oleksii Moisieiev
2025-06-30 12:13 ` Julien Grall
0 siblings, 1 reply; 67+ messages in thread
From: Oleksii Moisieiev @ 2025-06-30 11:57 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On 29/06/2025 21:34, Julien Grall wrote:
> Hi Oleksii,
>
> On 29/06/2025 16:41, Oleksii Moisieiev wrote:
> > --->
>> In the Virtualized system:
>
> Thanks for the long explanation. In this section, you mention Xen all
> over the place. But as you previously agreed the multi-agent SCMI is
> not Xen specific. To put it differently, aside the fact...
>
>>
> > - The Xen is real OSPM which manages other Virtual OSPM and it
> needs> dedicated SCMI management channel through which
>> it can perform HW resources assignment for Virtual OSPM by
>> communicating with EL2 SCMI FW
>> during Virtual OSPM creation or release HW resources during Virtual
>> OSPM destruction.
>> In the future it also possible to enable such PM feature as SCMI
>> based
>> CpuFreq in Xen.
>>
>> The SCMI DT binding for Xen SCMI Agent expected to be exactly the
>> same
>> as for any OSPM (like Linux) as from
>> SCMI FW point of few it is just OS running on Application Core (which
>> substitutes regular OS - Linux, RTOS).
>> So no new SCMI specific bindings (including compatibilities)
>> introduced neither in this series nor in this proposal.
>>
>> In other words, Xen needs two DT to boot, kinda:
>> - Xen DT (with bootinfo, Application Core info, uart)
>> - Host Platform DT (source information to create domains)
>> and if there would be two separate DTs - each will have own standard
>> (bindings) DT SCMI node.
>> According to the current design Xen accepts DT which is Host Platform DT
>> with added Xen configuration so Xen SCMI info is added in Xen
>> configuration node under /chosen, and no Domains is expected to see it
>> ever. After Xen initialization DT nodes from Xen DT are copied to the
>> Dom0 device tree. Our proposal is to keep SCMI configuration from
>> Platform Host DT the same so it will be copied to the Dom0 device tree.
>>
>>
>> - the number of Virtual Machines or Virtual OSPMs (in terms of SCMI)
>> depends on hypervisor (Xen) configuration.
>> And Virtual OSPM is defined as VM which has direct access to HW
>> (passthrough), so need access
>> SCMI services to get fine-grained and safe access to required
>> Platform
>> HW resources, and avoid interference.
>>
>> Every Virtual OSPM is SCMI agent, which sees it's own SCMI transport,
>> and doesn't know about other agents.
>> In the case of DT - the standard SCMI bindings are used.
>>
>> - So the Xen is the only entity in the platform which need to know about
>> other Agents.
>> Therefore, this Xen specific configuration
>> "xen,scmi-secondary-agents",
>> for the case of the EL2 SCMI FW, is introduced and added under the
>> /chosen node (or xen-config).
>> Unfortunately, there is no way to discover Agent's configurations
>> using SCMI protocol (base), like "func-id"
>> and shmem parameter (only can get Number of Agents, and discover own
>> Agent id), so only option is to
>> define this info in DT for Xen. However, Xen can use shared memory
>> regions and func_ids of the other Agents to determine agent_id using
>> base protocol. That's why it was decided to make agent_id in
>> "xem,scmi-secondary-agents" optional.
>
>
> ... the name of this property contains "xen", I still don't understand
> why the binding could not be used by other hypervisors. IOW, what if
> above you s/xen/KVM/ (or any other hypervisor you come up with)? Are
> they all going to create their own bindings? I would guess not given
> the single agent is already generic (if I am not mistaken, both Linux
> and Xen are using it)
>
KVM [1] is not applicable here as it starts under/inside Linux, so it
doesn't have direct access to SCMI, the Linux does.
And Linux will see only one SCMI transport (Agent).
Seems, the only option possible is virtio-scmi (qemu) - the virtio-scmi
potentially can simulate multi-channel,
but this is out if scope of this work.
QNX [0] relies on configuration files rather than the Device Tree.
[0]
https://www.qnx.com/developers/docs/8.0/com.qnx.doc.hypervisor.user/topic/config/hyp.html
[1]
https://trueconf.com/blog/knowledge-base/configure-kvm-hypervisor-ubuntu-server#KVM_configuration
> I will not insist on moving the binding outside of /chosen if the
> other maintainers think this is the best. But I think this is
> shortsighted to add "xen" in all the name or put it in a Xen specific
> position.
>
> Ultimately what I want to avoid is we have to support multiple
> bindings in Xen because someone else decided to create a new binding
> as we didn't even attempt to make ours generic...
>
Not aware of any similar task done or in progress.
> Cheers,
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-30 11:57 ` Oleksii Moisieiev
@ 2025-06-30 12:13 ` Julien Grall
2025-06-30 22:33 ` Stefano Stabellini
0 siblings, 1 reply; 67+ messages in thread
From: Julien Grall @ 2025-06-30 12:13 UTC (permalink / raw)
To: Oleksii Moisieiev, Stefano Stabellini
Cc: xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
Hi,
On 30/06/2025 12:57, Oleksii Moisieiev wrote:
> KVM [1] is not applicable here as it starts under/inside Linux, so it
> doesn't have direct access to SCMI, the Linux does.
> And Linux will see only one SCMI transport (Agent).
> Seems, the only option possible is virtio-scmi (qemu) - the virtio-scmi
> potentially can simulate multi-channel,
> but this is out if scope of this work.
> QNX [0] relies on configuration files rather than the Device Tree.
To clarify, I didn't ask to implement anything in KVM or QNX. I asked to
write the bindings in a way that they are not Xen specific to give a
chance to for other to re-use them. Yes today KVM and QNX would not use
them... But I also still don't see why we should actively prevent them
to use the bindings you come up with...
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH v4 8/8] docs: armproposa: l to add separate SCMI node for Xen agent
2025-06-30 12:13 ` Julien Grall
@ 2025-06-30 22:33 ` Stefano Stabellini
0 siblings, 0 replies; 67+ messages in thread
From: Stefano Stabellini @ 2025-06-30 22:33 UTC (permalink / raw)
To: Julien Grall
Cc: Oleksii Moisieiev, Stefano Stabellini,
xen-devel@lists.xenproject.org, Andrew Cooper, Anthony PERARD,
Bertrand Marquis, Jan Beulich, Juergen Gross, Michal Orzel,
Roger Pau Monné, Volodymyr Babchuk, Grygorii Strashko
On Mon, 30 Jun 2025, Julien Grall wrote:
> Hi,
>
> On 30/06/2025 12:57, Oleksii Moisieiev wrote:
> > KVM [1] is not applicable here as it starts under/inside Linux, so it
> > doesn't have direct access to SCMI, the Linux does.
> > And Linux will see only one SCMI transport (Agent).
> > Seems, the only option possible is virtio-scmi (qemu) - the virtio-scmi
> > potentially can simulate multi-channel,
> > but this is out if scope of this work.
> > QNX [0] relies on configuration files rather than the Device Tree.
> To clarify, I didn't ask to implement anything in KVM or QNX. I asked to write
> the bindings in a way that they are not Xen specific to give a chance to for
> other to re-use them. Yes today KVM and QNX would not use them... But I also
> still don't see why we should actively prevent them to use the bindings you
> come up with...
I agree with Julien. We can still have the property or node under
/chosen to avoid conflicts and confusion while still making the node
more generic so at least in theory it could be reused by other
hypervisors -- remove the "xen," prefix from
"xen,scmi-secondary-agents".
^ permalink raw reply [flat|nested] 67+ messages in thread
end of thread, other threads:[~2025-06-30 22:34 UTC | newest]
Thread overview: 67+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-05-19 15:50 [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 2/8] xen/arm: scmi-smc: update to be used under sci subsystem Oleksii Moisieiev
2025-05-19 23:29 ` Stefano Stabellini
2025-05-19 15:50 ` [RFC PATCH v4 1/8] xen/arm: add generic SCI subsystem Oleksii Moisieiev
2025-05-19 23:45 ` Stefano Stabellini
2025-05-19 15:50 ` [RFC PATCH v4 3/8] xen/arm: scmi-smc: passthrough SCMI SMC to domain, single agent Oleksii Moisieiev
2025-05-20 0:18 ` Stefano Stabellini
2025-05-19 15:50 ` [RFC PATCH v4 5/8] xen/domctl: extend XEN_DOMCTL_assign_device to handle not only iommu Oleksii Moisieiev
2025-05-19 18:54 ` Jan Beulich
2025-05-22 0:25 ` Stefano Stabellini
2025-05-22 6:18 ` Jan Beulich
2025-06-12 11:42 ` Oleksii Moisieiev
2025-06-12 13:10 ` Grygorii Strashko
2025-06-18 0:04 ` Stefano Stabellini
2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 21:30 ` Stefano Stabellini
2025-06-24 8:42 ` Oleksii Moisieiev
2025-06-24 8:47 ` Jan Beulich
2025-06-24 8:48 ` Oleksii Moisieiev
2025-06-23 7:15 ` Jan Beulich
2025-06-23 7:28 ` Oleksii Moisieiev
2025-06-25 19:56 ` Oleksii Moisieiev
2025-06-26 6:10 ` Jan Beulich
2025-06-26 13:07 ` Oleksii Moisieiev
2025-06-26 14:41 ` Jan Beulich
2025-06-26 15:01 ` Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 7/8] docs: arm: add SCI SCMI SMC multi-agent driver docs Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 4/8] docs: arm: add docs for SCMI over SMC calls forwarding driver Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 6/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent driver Oleksii Moisieiev
2025-05-23 20:06 ` Stefano Stabellini
2025-06-02 7:17 ` Bertrand Marquis
2025-06-12 12:03 ` Oleksii Moisieiev
2025-06-12 12:10 ` Grygorii Strashko
2025-06-17 23:38 ` Stefano Stabellini
2025-06-18 7:22 ` Julien Grall
2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 21:50 ` Stefano Stabellini
2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-05 22:44 ` Julien Grall
2025-06-12 12:03 ` Oleksii Moisieiev
2025-06-17 23:22 ` Stefano Stabellini
2025-06-17 23:56 ` Stefano Stabellini
2025-06-19 16:15 ` Oleksii Moisieiev
2025-06-22 22:15 ` Stefano Stabellini
2025-06-23 8:06 ` Julien Grall
2025-06-23 19:27 ` Stefano Stabellini
2025-06-23 20:38 ` Julien Grall
2025-06-23 20:42 ` Stefano Stabellini
2025-06-23 21:58 ` Julien Grall
2025-06-25 19:47 ` Oleksii Moisieiev
2025-06-18 7:28 ` Julien Grall
2025-06-19 16:15 ` Oleksii Moisieiev
2025-05-19 15:50 ` [RFC PATCH v4 8/8] docs: arm: proposal to add separate SCMI node for Xen agent Oleksii Moisieiev
2025-05-23 20:19 ` Stefano Stabellini
2025-06-12 12:03 ` Oleksii Moisieiev
2025-06-18 0:35 ` Stefano Stabellini
2025-06-19 16:15 ` [RFC PATCH v4 8/8] docs: armproposa: l " Oleksii Moisieiev
2025-06-22 21:57 ` Stefano Stabellini
2025-06-23 8:02 ` Julien Grall
2025-06-25 19:47 ` Oleksii Moisieiev
2025-06-25 20:32 ` Julien Grall
2025-06-29 15:41 ` Oleksii Moisieiev
2025-06-29 18:34 ` Julien Grall
2025-06-30 11:57 ` Oleksii Moisieiev
2025-06-30 12:13 ` Julien Grall
2025-06-30 22:33 ` Stefano Stabellini
2025-06-05 22:40 ` [RFC PATCH v4 0/8] xen/arm: scmi: introduce SCI SCMI SMC multi-agent support Julien Grall
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.