* [PATCH v4 0/3] Renesas RZ/Five add support to configure PMA and add vendor specific SBI extension
@ 2023-02-06 0:06 Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA Lad Prabhakar
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Lad Prabhakar @ 2023-02-06 0:06 UTC (permalink / raw)
To: opensbi
Hi All,
This patch series aims to add support to configure the PMA regions present
on the Andes AX45MP core (andes45-pma.c which can be shared with other
Andes vendors) and configures the required PMA region for RZ/Five SoC.
This patch series also adds the SBI vendor extension
RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to check if IOCP is present on
the core and if un-available it also checks if SW workaround for cache
management needs to be applied.
RFC v3 -> RFC v4
-> Used bit field for flags in struct andes45_pma_region
-> Renamed andes45-pma.h -> andes45_pma.h
-> Included RB tag from Lin san for patch 2/3
-> Moved CSR to platform/generic/include/andes/andes45.h
-> Included sbi/sbi_error.h in andes45-pma.c to fix build issue
RFC v2 -> RFC v3
-> Fixed review comments pointed by Lin-san
* Used switch case macro
* Added a check to verify the PMA size is power-of-2
* Dropped un-needed calculation done in PMA code for NAPOT.
* Dropped MISA macro
-> Fixed comments pointed by Himanshu
-> Fixed check patch issue
RFC v2 -> RFC v1
-> Fixed review comments pointed by Lin-san
-> Implemented PMA as library so that it can be shared
RFC v1:
https://patchwork.ozlabs.org/project/opensbi/patch/20221212094421.14556-1-prabhakar.mahadev-lad.rj at bp.renesas.com/
Cheers,
Prabhakar
Lad Prabhakar (3):
platform: generic: renesas: rzfive: Add support to configure the PMA
platform: generic: renesas: rzfive: Configure the PMA region
platform: generic: renesas: rzfive: Add SBI EXT to check for enabling
IOCP errata
platform/generic/Kconfig | 9 +
platform/generic/include/andes/andes45.h | 25 +-
platform/generic/include/andes45_pma.h | 48 +++
platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
platform/generic/renesas/rzfive/objects.mk | 1 +
platform/generic/renesas/rzfive/rzfive.c | 68 ++++
6 files changed, 499 insertions(+), 2 deletions(-)
create mode 100644 platform/generic/include/andes45_pma.h
create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c
--
2.17.1
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA
2023-02-06 0:06 [PATCH v4 0/3] Renesas RZ/Five add support to configure PMA and add vendor specific SBI extension Lad Prabhakar
@ 2023-02-06 0:06 ` Lad Prabhakar
2023-02-07 15:08 ` Yu-Chien Peter Lin
2023-02-13 4:07 ` Anup Patel
2023-02-06 0:06 ` [PATCH v4 2/3] platform: generic: renesas: rzfive: Configure the PMA region Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata Lad Prabhakar
2 siblings, 2 replies; 10+ messages in thread
From: Lad Prabhakar @ 2023-02-06 0:06 UTC (permalink / raw)
To: opensbi
I/O Coherence Port (IOCP) provides an AXI interface for connecting
external non-caching masters, such as DMA controllers. The accesses
from IOCP are coherent with D-Caches and L2 Cache.
IOCP is a specification option and is disabled on the Renesas RZ/Five
SoC due to this reason IP blocks using DMA will fail.
The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
block that allows dynamic adjustment of memory attributes in the runtime.
It contains a configurable amount of PMA entries implemented as CSR
registers to control the attributes of memory locations in interest.
Below are the memory attributes supported:
* Device, Non-bufferable
* Device, bufferable
* Memory, Non-cacheable, Non-bufferable
* Memory, Non-cacheable, Bufferable
* Memory, Write-back, No-allocate
* Memory, Write-back, Read-allocate
* Memory, Write-back, Write-allocate
* Memory, Write-back, Read and Write-allocate
More info about PMA (section 10.3):
Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
As a workaround for SoCs with IOCP disabled CMO needs to be handled by
software. Firstly OpenSBI configures the memory region as
"Memory, Non-cacheable, Bufferable" and passes this region as a global
shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
allocations happen from this region and synchronization callbacks are
implemented to synchronize when doing DMA transactions.
Example PMA region passed as a DT node from OpenSBI:
reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
ranges;
pma_resv0 at 58000000 {
compatible = "shared-dma-pool";
reg = <0x0 0x58000000 0x0 0x08000000>;
no-map;
linux,dma-default;
};
};
Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
---
platform/generic/Kconfig | 8 +
platform/generic/include/andes45_pma.h | 48 +++
platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
platform/generic/renesas/rzfive/objects.mk | 1 +
4 files changed, 407 insertions(+)
create mode 100644 platform/generic/include/andes45_pma.h
create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c
diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index c7f198a..9ee9fe3 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -51,3 +51,11 @@ config PLATFORM_STARFIVE_JH7110
default n
endif
+
+if PLATFORM_RENESAS_RZFIVE
+
+config ANDES45_PMA
+ bool "Andes PMA support"
+ default n
+
+endif
diff --git a/platform/generic/include/andes45_pma.h b/platform/generic/include/andes45_pma.h
new file mode 100644
index 0000000..37ec77c
--- /dev/null
+++ b/platform/generic/include/andes45_pma.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023 Renesas Electronics Corp.
+ */
+
+#ifndef _ANDES45_PMA_H_
+#define _ANDES45_PMA_H_
+
+#include <sbi/sbi_types.h>
+
+#define ANDES45_MAX_PMA_REGIONS 16
+
+/* Naturally aligned power of 2 region */
+#define ANDES45_PMACFG_ETYP_NAPOT 3
+
+/* Memory, Non-cacheable, Bufferable */
+#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF (3 << 2)
+
+/**
+ * struct andes45_pma_region - Describes PMA regions
+ *
+ * @pa: Address to be configured in the PMA
+ * @size: Size of the region
+ * @flags: Flags to be set for the PMA region
+ * @dt_populate: Boolean flag indicating if the DT entry should be
+ * populated for the given PMA region
+ * @shared_dma: Boolean flag if set "shared-dma-pool" property will
+ * be set in the DT node
+ * @no_map: Boolean flag if set "no-map" property will be set in the
+ * DT node
+ * @dma_default: Boolean flag if set "linux,dma-default" property will
+ * be set in the DT node. Note Linux expects single node
+ * with this property set.
+ */
+struct andes45_pma_region {
+ unsigned long pa;
+ unsigned long size;
+ u8 flags:7;
+ bool dt_populate;
+ bool shared_dma;
+ bool no_map;
+ bool dma_default;
+};
+
+int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
+ unsigned int pma_regions_count);
+
+#endif /* _ANDES45_PMA_H_ */
diff --git a/platform/generic/renesas/rzfive/andes45-pma.c b/platform/generic/renesas/rzfive/andes45-pma.c
new file mode 100644
index 0000000..881a628
--- /dev/null
+++ b/platform/generic/renesas/rzfive/andes45-pma.c
@@ -0,0 +1,350 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023 Renesas Electronics Corp.
+ *
+ * Copyright (c) 2020 Andes Technology Corporation
+ *
+ * Authors:
+ * Nick Hu <nickhu@andestech.com>
+ * Nylon Chen <nylon7@andestech.com>
+ * Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
+ */
+
+#include <andes45_pma.h>
+#include <libfdt.h>
+#include <sbi/riscv_asm.h>
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_console.h>
+#include <sbi/sbi_error.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+
+/* Configuration Registers */
+#define ANDES45_CSR_MMSC_CFG 0xFC2
+#define ANDES45_CSR_MMSC_PPMA_OFFSET (1 << 30)
+
+#define ANDES45_PMAADDR_0 0xBD0
+
+#define ANDES45_PMACFG_0 0xBC0
+
+static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
+{
+#define switchcase_pma_cfg_read(__pma_cfg_off, __val) \
+ case __pma_cfg_off: \
+ __val = csr_read(__pma_cfg_off); \
+ break;
+#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val) \
+ switchcase_pma_cfg_read(__pma_cfg_off + 0, __val) \
+ switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
+
+ unsigned long ret = 0;
+
+ switch (pma_cfg_off) {
+ switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
+
+ default:
+ sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
+ break;
+ }
+
+ return ret;
+
+#undef switchcase_pma_cfg_read_2
+#undef switchcase_pma_cfg_read
+}
+
+static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
+{
+#define switchcase_pma_cfg_write(__pma_cfg_off, __val) \
+ case __pma_cfg_off: \
+ csr_write(__pma_cfg_off, __val); \
+ break;
+#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val) \
+ switchcase_pma_cfg_write(__pma_cfg_off + 0, __val) \
+ switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
+
+ switch (pma_cfg_off) {
+ switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
+
+ default:
+ sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
+ break;
+ }
+
+#undef switchcase_pma_cfg_write_2
+#undef switchcase_pma_cfg_write
+}
+
+static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
+{
+#define switchcase_pma_write(__pma_addr_off, __val) \
+ case __pma_addr_off: \
+ csr_write(__pma_addr_off, __val); \
+ break;
+#define switchcase_pma_write_2(__pma_addr_off, __val) \
+ switchcase_pma_write(__pma_addr_off + 0, __val) \
+ switchcase_pma_write(__pma_addr_off + 1, __val)
+#define switchcase_pma_write_4(__pma_addr_off, __val) \
+ switchcase_pma_write_2(__pma_addr_off + 0, __val) \
+ switchcase_pma_write_2(__pma_addr_off + 2, __val)
+#define switchcase_pma_write_8(__pma_addr_off, __val) \
+ switchcase_pma_write_4(__pma_addr_off + 0, __val) \
+ switchcase_pma_write_4(__pma_addr_off + 4, __val)
+#define switchcase_pma_write_16(__pma_addr_off, __val) \
+ switchcase_pma_write_8(__pma_addr_off + 0, __val) \
+ switchcase_pma_write_8(__pma_addr_off + 8, __val)
+
+ switch (pma_addr_off) {
+ switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
+
+ default:
+ sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
+ break;
+ }
+
+#undef switchcase_pma_write_16
+#undef switchcase_pma_write_8
+#undef switchcase_pma_write_4
+#undef switchcase_pma_write_2
+#undef switchcase_pma_write
+}
+
+static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
+{
+#define switchcase_pma_read(__pma_addr_off, __val) \
+ case __pma_addr_off: \
+ __val = csr_read(__pma_addr_off); \
+ break;
+#define switchcase_pma_read_2(__pma_addr_off, __val) \
+ switchcase_pma_read(__pma_addr_off + 0, __val) \
+ switchcase_pma_read(__pma_addr_off + 1, __val)
+#define switchcase_pma_read_4(__pma_addr_off, __val) \
+ switchcase_pma_read_2(__pma_addr_off + 0, __val) \
+ switchcase_pma_read_2(__pma_addr_off + 2, __val)
+#define switchcase_pma_read_8(__pma_addr_off, __val) \
+ switchcase_pma_read_4(__pma_addr_off + 0, __val) \
+ switchcase_pma_read_4(__pma_addr_off + 4, __val)
+#define switchcase_pma_read_16(__pma_addr_off, __val) \
+ switchcase_pma_read_8(__pma_addr_off + 0, __val) \
+ switchcase_pma_read_8(__pma_addr_off + 8, __val)
+
+ unsigned long ret = 0;
+
+ switch (pma_addr_off) {
+ switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
+
+ default:
+ sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
+ break;
+ }
+
+ return ret;
+
+#undef switchcase_pma_read_16
+#undef switchcase_pma_read_8
+#undef switchcase_pma_read_4
+#undef switchcase_pma_read_2
+#undef switchcase_pma_read
+}
+
+static unsigned long
+andes45_pma_setup(const struct andes45_pma_region *pma_region,
+ unsigned int entry_id)
+{
+ unsigned long size = pma_region->size;
+ unsigned long addr = pma_region->pa;
+ unsigned int pma_cfg_addr;
+ unsigned long pmacfg_val;
+ unsigned long pmaaddr;
+ char *pmaxcfg;
+
+ /* Check for 4KiB granularity */
+ if (size < (1 << 12))
+ return SBI_EINVAL;
+
+ /* Check size is power of 2 */
+ if (size & (size - 1))
+ return SBI_EINVAL;
+
+ if (entry_id > 15)
+ return SBI_EINVAL;
+
+ if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
+ return SBI_EINVAL;
+
+ if ((addr & (size - 1)) != 0)
+ return SBI_EINVAL;
+
+ pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
+ pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
+ pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
+ *pmaxcfg = 0;
+ *pmaxcfg = pma_region->flags;
+
+ andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
+
+ pmaaddr = (addr >> 2) + (size >> 3) - 1;
+
+ andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
+
+ return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
+ pmaaddr : SBI_EINVAL;
+}
+
+static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
+ unsigned int index, int parent)
+{
+ int na = fdt_address_cells(fdt, 0);
+ int ns = fdt_size_cells(fdt, 0);
+ static bool dma_default = false;
+ fdt32_t addr_high, addr_low;
+ fdt32_t size_high, size_low;
+ int subnode, err;
+ fdt32_t reg[4];
+ fdt32_t *val;
+ char name[32];
+
+ addr_high = (u64)pma->pa >> 32;
+ addr_low = pma->pa;
+ size_high = (u64)pma->size >> 32;
+ size_low = pma->size;
+
+ if (na > 1 && addr_high)
+ sbi_snprintf(name, sizeof(name),
+ "pma_resv%d@%x,%x", index,
+ addr_high, addr_low);
+ else
+ sbi_snprintf(name, sizeof(name),
+ "pma_resv%d@%x", index,
+ addr_low);
+
+ subnode = fdt_add_subnode(fdt, parent, name);
+ if (subnode < 0)
+ return subnode;
+
+ if (pma->shared_dma) {
+ err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
+ if (err < 0)
+ return err;
+ }
+
+ if (pma->no_map) {
+ err = fdt_setprop_empty(fdt, subnode, "no-map");
+ if (err < 0)
+ return err;
+ }
+
+ /* Linux allows single linux,dma-default region. */
+ if (pma->dma_default) {
+ if (dma_default)
+ return SBI_EINVAL;
+
+ err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
+ if (err < 0)
+ return err;
+ dma_default = true;
+ }
+
+ /* encode the <reg> property value */
+ val = reg;
+ if (na > 1)
+ *val++ = cpu_to_fdt32(addr_high);
+ *val++ = cpu_to_fdt32(addr_low);
+ if (ns > 1)
+ *val++ = cpu_to_fdt32(size_high);
+ *val++ = cpu_to_fdt32(size_low);
+
+ err = fdt_setprop(fdt, subnode, "reg", reg,
+ (na + ns) * sizeof(fdt32_t));
+ if (err < 0)
+ return err;
+
+ return 0;
+}
+
+static int andes45_fdt_reserved_memory_fixup(void *fdt,
+ const struct andes45_pma_region *pma,
+ unsigned int entry)
+{
+ int parent;
+
+ /* try to locate the reserved memory node */
+ parent = fdt_path_offset(fdt, "/reserved-memory");
+ if (parent < 0) {
+ int na = fdt_address_cells(fdt, 0);
+ int ns = fdt_size_cells(fdt, 0);
+ int err;
+
+ /* if such node does not exist, create one */
+ parent = fdt_add_subnode(fdt, 0, "reserved-memory");
+ if (parent < 0)
+ return parent;
+
+ err = fdt_setprop_empty(fdt, parent, "ranges");
+ if (err < 0)
+ return err;
+
+ err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
+ if (err < 0)
+ return err;
+
+ err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
+ if (err < 0)
+ return err;
+ }
+
+ return andes45_fdt_pma_resv(fdt, pma, entry, parent);
+}
+
+int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
+ unsigned int pma_regions_count)
+{
+ unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
+ unsigned int dt_populate_cnt;
+ unsigned int i, j;
+ unsigned long pa;
+ void *fdt;
+ int ret;
+
+ if (!pma_regions || !pma_regions_count)
+ return 0;
+
+ if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
+ return SBI_EINVAL;
+
+ if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
+ return SBI_ENOTSUPP;
+
+ /* Configure the PMA regions */
+ for (i = 0; i < pma_regions_count; i++) {
+ pa = andes45_pma_setup(&pma_regions[i], i);
+ if (pa == SBI_EINVAL)
+ return SBI_EINVAL;
+ }
+
+ dt_populate_cnt = 0;
+ for (i = 0; i < pma_regions_count; i++) {
+ if (!pma_regions[i].dt_populate)
+ continue;
+ dt_populate_cnt++;
+ }
+
+ if (!dt_populate_cnt)
+ return 0;
+
+ fdt = fdt_get_address();
+
+ ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
+ if (ret < 0)
+ return ret;
+
+ for (i = 0, j = 0; i < pma_regions_count; i++) {
+ if (!pma_regions[i].dt_populate)
+ continue;
+
+ ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
diff --git a/platform/generic/renesas/rzfive/objects.mk b/platform/generic/renesas/rzfive/objects.mk
index 2e7e37f..fd9e7c4 100644
--- a/platform/generic/renesas/rzfive/objects.mk
+++ b/platform/generic/renesas/rzfive/objects.mk
@@ -6,3 +6,4 @@
carray-platform_override_modules-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas_rzfive
platform-objs-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas/rzfive/rzfive.o
+platform-objs-$(CONFIG_ANDES45_PMA) += renesas/rzfive/andes45-pma.o
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 2/3] platform: generic: renesas: rzfive: Configure the PMA region
2023-02-06 0:06 [PATCH v4 0/3] Renesas RZ/Five add support to configure PMA and add vendor specific SBI extension Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA Lad Prabhakar
@ 2023-02-06 0:06 ` Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata Lad Prabhakar
2 siblings, 0 replies; 10+ messages in thread
From: Lad Prabhakar @ 2023-02-06 0:06 UTC (permalink / raw)
To: opensbi
On the Renesas RZ/Five SoC by default we want to configure 128MiB of memory
ranging from 0x58000000 as a non-cacheable + bufferable region in the PMA
and populate this region as PMA reserve DT node with shared DMA pool and
no-map flags set so that Linux drivers requesting any DMA'able memory go
through this region.
PMA node passed to the above stack:
reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
ranges;
pma_resv0 at 58000000 {
compatible = "shared-dma-pool";
reg = <0x0 0x58000000 0x0 0x08000000>;
no-map;
linux,dma-default;
};
};
Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
---
platform/generic/Kconfig | 1 +
platform/generic/renesas/rzfive/rzfive.c | 21 +++++++++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index 9ee9fe3..bc5c280 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -35,6 +35,7 @@ config PLATFORM_ANDES_AE350
config PLATFORM_RENESAS_RZFIVE
bool "Renesas RZ/Five support"
+ select ANDES45_PMA
default n
config PLATFORM_SIFIVE_FU540
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index ee9c9c4..eee9c51 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -4,10 +4,30 @@
*
*/
+#include <andes45_pma.h>
#include <platform_override.h>
#include <sbi/sbi_domain.h>
#include <sbi_utils/fdt/fdt_helper.h>
+static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
+ {
+ .pa = 0x58000000,
+ .size = 0x8000000,
+ .flags = ANDES45_PMACFG_ETYP_NAPOT |
+ ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF,
+ .dt_populate = true,
+ .shared_dma = true,
+ .no_map = true,
+ .dma_default = true,
+ },
+};
+
+static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *match)
+{
+ return andes45_pma_setup_regions(renesas_rzfive_pma_regions,
+ array_size(renesas_rzfive_pma_regions));
+}
+
int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
{
/*
@@ -34,4 +54,5 @@ static const struct fdt_match renesas_rzfive_match[] = {
const struct platform_override renesas_rzfive = {
.match_table = renesas_rzfive_match,
.early_init = renesas_rzfive_early_init,
+ .final_init = renesas_rzfive_final_init,
};
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata
2023-02-06 0:06 [PATCH v4 0/3] Renesas RZ/Five add support to configure PMA and add vendor specific SBI extension Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 2/3] platform: generic: renesas: rzfive: Configure the PMA region Lad Prabhakar
@ 2023-02-06 0:06 ` Lad Prabhakar
2023-02-07 15:21 ` Yu-Chien Peter Lin
2023-02-13 4:15 ` Anup Patel
2 siblings, 2 replies; 10+ messages in thread
From: Lad Prabhakar @ 2023-02-06 0:06 UTC (permalink / raw)
To: opensbi
I/O Coherence Port (IOCP) provides an AXI interface for connecting
external non-caching masters, such as DMA controllers. The accesses
from IOCP are coherent with D-Caches and L2 Cache.
IOCP is a specification option and is disabled on the Renesas RZ/Five
SoC due to this reason IP blocks using DMA will fail.
As a workaround for SoCs with IOCP disabled CMO needs to be handled by
software. Firstly OpenSBI configures the memory region as
"Memory, Non-cacheable, Bufferable" and passes this region as a global
shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
allocations happen from this region and synchronization callbacks are
implemented to synchronize when doing DMA transactions.
RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
errata should be applied to handle cache management.
Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
---
platform/generic/include/andes/andes45.h | 25 ++++++++++++-
platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
2 files changed, 70 insertions(+), 2 deletions(-)
diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
index 08b3d18..d5d265f 100644
--- a/platform/generic/include/andes/andes45.h
+++ b/platform/generic/include/andes/andes45.h
@@ -4,7 +4,28 @@
#define CSR_MARCHID_MICROID 0xfff
/* Memory and Miscellaneous Registers */
-#define CSR_MCACHE_CTL 0x7ca
-#define CSR_MCCTLCOMMAND 0x7cc
+#define CSR_MCACHE_CTL 0x7ca
+#define CSR_MCCTLCOMMAND 0x7cc
+#define CSR_MICM_CFG 0xfc0
+#define CSR_MDCM_CFG 0xfc1
+#define CSR_MMSC_CFG 0xfc2
+
+#define MISA_20_OFFSET 20
+#define MISA_20_MASK (0x1 << MISA_20_OFFSET)
+
+#define MICM_CFG_ISZ_OFFSET 6
+#define MICM_CFG_ISZ_MASK (0x7 << MICM_CFG_ISZ_OFFSET)
+
+#define MDCM_CFG_DSZ_OFFSET 6
+#define MDCM_CFG_DSZ_MASK (0x7 << MDCM_CFG_DSZ_OFFSET)
+
+#define MMSC_CFG_CCTLCSR_OFFSET 16
+#define MMSC_CFG_CCTLCSR_MASK (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
+#define MMSC_IOCP_OFFSET 47
+#define MMSC_IOCP_MASK (0x1ULL << MMSC_IOCP_OFFSET)
+
+#define MCACHE_CTL_CCTL_SUEN_OFFSET 8
+#define MCACHE_CTL_CCTL_SUEN_MASK (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
+
#endif /* _RISCV_ANDES45_H */
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index eee9c51..552c747 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -4,11 +4,17 @@
*
*/
+#include <andes/andes45.h>
#include <andes45_pma.h>
#include <platform_override.h>
#include <sbi/sbi_domain.h>
+#include <sbi/sbi_error.h>
#include <sbi_utils/fdt/fdt_helper.h>
+#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND 0
+
+#define ANDESTECH_SBI_EXT_ANDES 0x900031e
+
static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
{
.pa = 0x58000000,
@@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
array_size(renesas_rzfive_pma_regions));
}
+static bool renesas_rzfive_cpu_cache_controlable(void)
+{
+ return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
+ (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
+ (csr_read(CSR_MISA) & MISA_20_MASK) &&
+ (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
+ (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
+}
+
+static bool renesas_rzfive_cpu_iocp_disabled(void)
+{
+ return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
+}
+
+static bool renesas_rzfive_apply_iocp_sw_workaround(void)
+{
+ return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
+}
+
+static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
+ const struct sbi_trap_regs *regs,
+ unsigned long *out_value,
+ struct sbi_trap_info *out_trap,
+ const struct fdt_match *match)
+{
+ if (extid != ANDESTECH_SBI_EXT_ANDES)
+ return SBI_EINVAL;
+
+ switch (funcid) {
+ case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
+ *out_value = renesas_rzfive_apply_iocp_sw_workaround();
+ break;
+
+ default:
+ break;
+ }
+
+ return 0;
+}
+
int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
{
/*
@@ -55,4 +101,5 @@ const struct platform_override renesas_rzfive = {
.match_table = renesas_rzfive_match,
.early_init = renesas_rzfive_early_init,
.final_init = renesas_rzfive_final_init,
+ .vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
};
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA
2023-02-06 0:06 ` [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA Lad Prabhakar
@ 2023-02-07 15:08 ` Yu-Chien Peter Lin
2023-02-13 4:07 ` Anup Patel
1 sibling, 0 replies; 10+ messages in thread
From: Yu-Chien Peter Lin @ 2023-02-07 15:08 UTC (permalink / raw)
To: opensbi
On Mon, Feb 06, 2023 at 12:06:22AM +0000, Lad Prabhakar wrote:
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> block that allows dynamic adjustment of memory attributes in the runtime.
> It contains a configurable amount of PMA entries implemented as CSR
> registers to control the attributes of memory locations in interest.
> Below are the memory attributes supported:
> * Device, Non-bufferable
> * Device, bufferable
> * Memory, Non-cacheable, Non-bufferable
> * Memory, Non-cacheable, Bufferable
> * Memory, Write-back, No-allocate
> * Memory, Write-back, Read-allocate
> * Memory, Write-back, Write-allocate
> * Memory, Write-back, Read and Write-allocate
>
> More info about PMA (section 10.3):
> Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> Example PMA region passed as a DT node from OpenSBI:
> reserved-memory {
> #address-cells = <2>;
> #size-cells = <2>;
> ranges;
>
> pma_resv0 at 58000000 {
> compatible = "shared-dma-pool";
> reg = <0x0 0x58000000 0x0 0x08000000>;
> no-map;
> linux,dma-default;
> };
> };
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Regards,
Peter Lin
> ---
> platform/generic/Kconfig | 8 +
> platform/generic/include/andes45_pma.h | 48 +++
> platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
> platform/generic/renesas/rzfive/objects.mk | 1 +
> 4 files changed, 407 insertions(+)
> create mode 100644 platform/generic/include/andes45_pma.h
> create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index c7f198a..9ee9fe3 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -51,3 +51,11 @@ config PLATFORM_STARFIVE_JH7110
> default n
>
> endif
> +
> +if PLATFORM_RENESAS_RZFIVE
> +
> +config ANDES45_PMA
> + bool "Andes PMA support"
> + default n
> +
> +endif
> diff --git a/platform/generic/include/andes45_pma.h b/platform/generic/include/andes45_pma.h
> new file mode 100644
> index 0000000..37ec77c
> --- /dev/null
> +++ b/platform/generic/include/andes45_pma.h
> @@ -0,0 +1,48 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + */
> +
> +#ifndef _ANDES45_PMA_H_
> +#define _ANDES45_PMA_H_
> +
> +#include <sbi/sbi_types.h>
> +
> +#define ANDES45_MAX_PMA_REGIONS 16
> +
> +/* Naturally aligned power of 2 region */
> +#define ANDES45_PMACFG_ETYP_NAPOT 3
> +
> +/* Memory, Non-cacheable, Bufferable */
> +#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF (3 << 2)
> +
> +/**
> + * struct andes45_pma_region - Describes PMA regions
> + *
> + * @pa: Address to be configured in the PMA
> + * @size: Size of the region
> + * @flags: Flags to be set for the PMA region
> + * @dt_populate: Boolean flag indicating if the DT entry should be
> + * populated for the given PMA region
> + * @shared_dma: Boolean flag if set "shared-dma-pool" property will
> + * be set in the DT node
> + * @no_map: Boolean flag if set "no-map" property will be set in the
> + * DT node
> + * @dma_default: Boolean flag if set "linux,dma-default" property will
> + * be set in the DT node. Note Linux expects single node
> + * with this property set.
> + */
> +struct andes45_pma_region {
> + unsigned long pa;
> + unsigned long size;
> + u8 flags:7;
> + bool dt_populate;
> + bool shared_dma;
> + bool no_map;
> + bool dma_default;
> +};
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> + unsigned int pma_regions_count);
> +
> +#endif /* _ANDES45_PMA_H_ */
> diff --git a/platform/generic/renesas/rzfive/andes45-pma.c b/platform/generic/renesas/rzfive/andes45-pma.c
> new file mode 100644
> index 0000000..881a628
> --- /dev/null
> +++ b/platform/generic/renesas/rzfive/andes45-pma.c
> @@ -0,0 +1,350 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + *
> + * Copyright (c) 2020 Andes Technology Corporation
> + *
> + * Authors:
> + * Nick Hu <nickhu@andestech.com>
> + * Nylon Chen <nylon7@andestech.com>
> + * Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> + */
> +
> +#include <andes45_pma.h>
> +#include <libfdt.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +
> +/* Configuration Registers */
> +#define ANDES45_CSR_MMSC_CFG 0xFC2
> +#define ANDES45_CSR_MMSC_PPMA_OFFSET (1 << 30)
> +
> +#define ANDES45_PMAADDR_0 0xBD0
> +
> +#define ANDES45_PMACFG_0 0xBC0
> +
> +static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
> +{
> +#define switchcase_pma_cfg_read(__pma_cfg_off, __val) \
> + case __pma_cfg_off: \
> + __val = csr_read(__pma_cfg_off); \
> + break;
> +#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val) \
> + switchcase_pma_cfg_read(__pma_cfg_off + 0, __val) \
> + switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
> +
> + unsigned long ret = 0;
> +
> + switch (pma_cfg_off) {
> + switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
> +
> + default:
> + sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> + break;
> + }
> +
> + return ret;
> +
> +#undef switchcase_pma_cfg_read_2
> +#undef switchcase_pma_cfg_read
> +}
> +
> +static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
> +{
> +#define switchcase_pma_cfg_write(__pma_cfg_off, __val) \
> + case __pma_cfg_off: \
> + csr_write(__pma_cfg_off, __val); \
> + break;
> +#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val) \
> + switchcase_pma_cfg_write(__pma_cfg_off + 0, __val) \
> + switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
> +
> + switch (pma_cfg_off) {
> + switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
> +
> + default:
> + sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> + break;
> + }
> +
> +#undef switchcase_pma_cfg_write_2
> +#undef switchcase_pma_cfg_write
> +}
> +
> +static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
> +{
> +#define switchcase_pma_write(__pma_addr_off, __val) \
> + case __pma_addr_off: \
> + csr_write(__pma_addr_off, __val); \
> + break;
> +#define switchcase_pma_write_2(__pma_addr_off, __val) \
> + switchcase_pma_write(__pma_addr_off + 0, __val) \
> + switchcase_pma_write(__pma_addr_off + 1, __val)
> +#define switchcase_pma_write_4(__pma_addr_off, __val) \
> + switchcase_pma_write_2(__pma_addr_off + 0, __val) \
> + switchcase_pma_write_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_write_8(__pma_addr_off, __val) \
> + switchcase_pma_write_4(__pma_addr_off + 0, __val) \
> + switchcase_pma_write_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_write_16(__pma_addr_off, __val) \
> + switchcase_pma_write_8(__pma_addr_off + 0, __val) \
> + switchcase_pma_write_8(__pma_addr_off + 8, __val)
> +
> + switch (pma_addr_off) {
> + switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
> +
> + default:
> + sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> + break;
> + }
> +
> +#undef switchcase_pma_write_16
> +#undef switchcase_pma_write_8
> +#undef switchcase_pma_write_4
> +#undef switchcase_pma_write_2
> +#undef switchcase_pma_write
> +}
> +
> +static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
> +{
> +#define switchcase_pma_read(__pma_addr_off, __val) \
> + case __pma_addr_off: \
> + __val = csr_read(__pma_addr_off); \
> + break;
> +#define switchcase_pma_read_2(__pma_addr_off, __val) \
> + switchcase_pma_read(__pma_addr_off + 0, __val) \
> + switchcase_pma_read(__pma_addr_off + 1, __val)
> +#define switchcase_pma_read_4(__pma_addr_off, __val) \
> + switchcase_pma_read_2(__pma_addr_off + 0, __val) \
> + switchcase_pma_read_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_read_8(__pma_addr_off, __val) \
> + switchcase_pma_read_4(__pma_addr_off + 0, __val) \
> + switchcase_pma_read_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_read_16(__pma_addr_off, __val) \
> + switchcase_pma_read_8(__pma_addr_off + 0, __val) \
> + switchcase_pma_read_8(__pma_addr_off + 8, __val)
> +
> + unsigned long ret = 0;
> +
> + switch (pma_addr_off) {
> + switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
> +
> + default:
> + sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> + break;
> + }
> +
> + return ret;
> +
> +#undef switchcase_pma_read_16
> +#undef switchcase_pma_read_8
> +#undef switchcase_pma_read_4
> +#undef switchcase_pma_read_2
> +#undef switchcase_pma_read
> +}
> +
> +static unsigned long
> +andes45_pma_setup(const struct andes45_pma_region *pma_region,
> + unsigned int entry_id)
> +{
> + unsigned long size = pma_region->size;
> + unsigned long addr = pma_region->pa;
> + unsigned int pma_cfg_addr;
> + unsigned long pmacfg_val;
> + unsigned long pmaaddr;
> + char *pmaxcfg;
> +
> + /* Check for 4KiB granularity */
> + if (size < (1 << 12))
> + return SBI_EINVAL;
> +
> + /* Check size is power of 2 */
> + if (size & (size - 1))
> + return SBI_EINVAL;
> +
> + if (entry_id > 15)
> + return SBI_EINVAL;
> +
> + if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
> + return SBI_EINVAL;
> +
> + if ((addr & (size - 1)) != 0)
> + return SBI_EINVAL;
> +
> + pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
> + pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
> + pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
> + *pmaxcfg = 0;
> + *pmaxcfg = pma_region->flags;
> +
> + andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
> +
> + pmaaddr = (addr >> 2) + (size >> 3) - 1;
> +
> + andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
> +
> + return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
> + pmaaddr : SBI_EINVAL;
> +}
> +
> +static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
> + unsigned int index, int parent)
> +{
> + int na = fdt_address_cells(fdt, 0);
> + int ns = fdt_size_cells(fdt, 0);
> + static bool dma_default = false;
> + fdt32_t addr_high, addr_low;
> + fdt32_t size_high, size_low;
> + int subnode, err;
> + fdt32_t reg[4];
> + fdt32_t *val;
> + char name[32];
> +
> + addr_high = (u64)pma->pa >> 32;
> + addr_low = pma->pa;
> + size_high = (u64)pma->size >> 32;
> + size_low = pma->size;
> +
> + if (na > 1 && addr_high)
> + sbi_snprintf(name, sizeof(name),
> + "pma_resv%d@%x,%x", index,
> + addr_high, addr_low);
> + else
> + sbi_snprintf(name, sizeof(name),
> + "pma_resv%d@%x", index,
> + addr_low);
> +
> + subnode = fdt_add_subnode(fdt, parent, name);
> + if (subnode < 0)
> + return subnode;
> +
> + if (pma->shared_dma) {
> + err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
> + if (err < 0)
> + return err;
> + }
> +
> + if (pma->no_map) {
> + err = fdt_setprop_empty(fdt, subnode, "no-map");
> + if (err < 0)
> + return err;
> + }
> +
> + /* Linux allows single linux,dma-default region. */
> + if (pma->dma_default) {
> + if (dma_default)
> + return SBI_EINVAL;
> +
> + err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
> + if (err < 0)
> + return err;
> + dma_default = true;
> + }
> +
> + /* encode the <reg> property value */
> + val = reg;
> + if (na > 1)
> + *val++ = cpu_to_fdt32(addr_high);
> + *val++ = cpu_to_fdt32(addr_low);
> + if (ns > 1)
> + *val++ = cpu_to_fdt32(size_high);
> + *val++ = cpu_to_fdt32(size_low);
> +
> + err = fdt_setprop(fdt, subnode, "reg", reg,
> + (na + ns) * sizeof(fdt32_t));
> + if (err < 0)
> + return err;
> +
> + return 0;
> +}
> +
> +static int andes45_fdt_reserved_memory_fixup(void *fdt,
> + const struct andes45_pma_region *pma,
> + unsigned int entry)
> +{
> + int parent;
> +
> + /* try to locate the reserved memory node */
> + parent = fdt_path_offset(fdt, "/reserved-memory");
> + if (parent < 0) {
> + int na = fdt_address_cells(fdt, 0);
> + int ns = fdt_size_cells(fdt, 0);
> + int err;
> +
> + /* if such node does not exist, create one */
> + parent = fdt_add_subnode(fdt, 0, "reserved-memory");
> + if (parent < 0)
> + return parent;
> +
> + err = fdt_setprop_empty(fdt, parent, "ranges");
> + if (err < 0)
> + return err;
> +
> + err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
> + if (err < 0)
> + return err;
> +
> + err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
> + if (err < 0)
> + return err;
> + }
> +
> + return andes45_fdt_pma_resv(fdt, pma, entry, parent);
> +}
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> + unsigned int pma_regions_count)
> +{
> + unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
> + unsigned int dt_populate_cnt;
> + unsigned int i, j;
> + unsigned long pa;
> + void *fdt;
> + int ret;
> +
> + if (!pma_regions || !pma_regions_count)
> + return 0;
> +
> + if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
> + return SBI_EINVAL;
> +
> + if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
> + return SBI_ENOTSUPP;
> +
> + /* Configure the PMA regions */
> + for (i = 0; i < pma_regions_count; i++) {
> + pa = andes45_pma_setup(&pma_regions[i], i);
> + if (pa == SBI_EINVAL)
> + return SBI_EINVAL;
> + }
> +
> + dt_populate_cnt = 0;
> + for (i = 0; i < pma_regions_count; i++) {
> + if (!pma_regions[i].dt_populate)
> + continue;
> + dt_populate_cnt++;
> + }
> +
> + if (!dt_populate_cnt)
> + return 0;
> +
> + fdt = fdt_get_address();
> +
> + ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
> + if (ret < 0)
> + return ret;
> +
> + for (i = 0, j = 0; i < pma_regions_count; i++) {
> + if (!pma_regions[i].dt_populate)
> + continue;
> +
> + ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> diff --git a/platform/generic/renesas/rzfive/objects.mk b/platform/generic/renesas/rzfive/objects.mk
> index 2e7e37f..fd9e7c4 100644
> --- a/platform/generic/renesas/rzfive/objects.mk
> +++ b/platform/generic/renesas/rzfive/objects.mk
> @@ -6,3 +6,4 @@
>
> carray-platform_override_modules-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas_rzfive
> platform-objs-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas/rzfive/rzfive.o
> +platform-objs-$(CONFIG_ANDES45_PMA) += renesas/rzfive/andes45-pma.o
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata
2023-02-06 0:06 ` [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata Lad Prabhakar
@ 2023-02-07 15:21 ` Yu-Chien Peter Lin
2023-02-13 4:15 ` Anup Patel
1 sibling, 0 replies; 10+ messages in thread
From: Yu-Chien Peter Lin @ 2023-02-07 15:21 UTC (permalink / raw)
To: opensbi
On Mon, Feb 06, 2023 at 12:06:24AM +0000, Lad Prabhakar wrote:
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> errata should be applied to handle cache management.
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Regards,
Peter Lin
> ---
> platform/generic/include/andes/andes45.h | 25 ++++++++++++-
> platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
> 2 files changed, 70 insertions(+), 2 deletions(-)
>
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index 08b3d18..d5d265f 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -4,7 +4,28 @@
> #define CSR_MARCHID_MICROID 0xfff
>
> /* Memory and Miscellaneous Registers */
> -#define CSR_MCACHE_CTL 0x7ca
> -#define CSR_MCCTLCOMMAND 0x7cc
> +#define CSR_MCACHE_CTL 0x7ca
> +#define CSR_MCCTLCOMMAND 0x7cc
> +#define CSR_MICM_CFG 0xfc0
> +#define CSR_MDCM_CFG 0xfc1
> +#define CSR_MMSC_CFG 0xfc2
> +
> +#define MISA_20_OFFSET 20
> +#define MISA_20_MASK (0x1 << MISA_20_OFFSET)
> +
> +#define MICM_CFG_ISZ_OFFSET 6
> +#define MICM_CFG_ISZ_MASK (0x7 << MICM_CFG_ISZ_OFFSET)
> +
> +#define MDCM_CFG_DSZ_OFFSET 6
> +#define MDCM_CFG_DSZ_MASK (0x7 << MDCM_CFG_DSZ_OFFSET)
> +
> +#define MMSC_CFG_CCTLCSR_OFFSET 16
> +#define MMSC_CFG_CCTLCSR_MASK (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> +#define MMSC_IOCP_OFFSET 47
> +#define MMSC_IOCP_MASK (0x1ULL << MMSC_IOCP_OFFSET)
> +
> +#define MCACHE_CTL_CCTL_SUEN_OFFSET 8
> +#define MCACHE_CTL_CCTL_SUEN_MASK (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> +
>
> #endif /* _RISCV_ANDES45_H */
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index eee9c51..552c747 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -4,11 +4,17 @@
> *
> */
>
> +#include <andes/andes45.h>
> #include <andes45_pma.h>
> #include <platform_override.h>
> #include <sbi/sbi_domain.h>
> +#include <sbi/sbi_error.h>
> #include <sbi_utils/fdt/fdt_helper.h>
>
> +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND 0
> +
> +#define ANDESTECH_SBI_EXT_ANDES 0x900031e
> +
> static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
> {
> .pa = 0x58000000,
> @@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
> array_size(renesas_rzfive_pma_regions));
> }
>
> +static bool renesas_rzfive_cpu_cache_controlable(void)
> +{
> + return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
> + (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
> + (csr_read(CSR_MISA) & MISA_20_MASK) &&
> + (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
> + (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
> +}
> +
> +static bool renesas_rzfive_cpu_iocp_disabled(void)
> +{
> + return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
> +}
> +
> +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
> +{
> + return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
> +}
> +
> +static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
> + const struct sbi_trap_regs *regs,
> + unsigned long *out_value,
> + struct sbi_trap_info *out_trap,
> + const struct fdt_match *match)
> +{
> + if (extid != ANDESTECH_SBI_EXT_ANDES)
> + return SBI_EINVAL;
> +
> + switch (funcid) {
> + case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
> + *out_value = renesas_rzfive_apply_iocp_sw_workaround();
> + break;
> +
> + default:
> + break;
> + }
> +
> + return 0;
> +}
> +
> int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
> {
> /*
> @@ -55,4 +101,5 @@ const struct platform_override renesas_rzfive = {
> .match_table = renesas_rzfive_match,
> .early_init = renesas_rzfive_early_init,
> .final_init = renesas_rzfive_final_init,
> + .vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
> };
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA
2023-02-06 0:06 ` [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA Lad Prabhakar
2023-02-07 15:08 ` Yu-Chien Peter Lin
@ 2023-02-13 4:07 ` Anup Patel
2023-02-13 21:46 ` Lad, Prabhakar
1 sibling, 1 reply; 10+ messages in thread
From: Anup Patel @ 2023-02-13 4:07 UTC (permalink / raw)
To: opensbi
On Mon, Feb 6, 2023 at 5:36 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj@bp.renesas.com> wrote:
>
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> block that allows dynamic adjustment of memory attributes in the runtime.
> It contains a configurable amount of PMA entries implemented as CSR
> registers to control the attributes of memory locations in interest.
> Below are the memory attributes supported:
> * Device, Non-bufferable
> * Device, bufferable
> * Memory, Non-cacheable, Non-bufferable
> * Memory, Non-cacheable, Bufferable
> * Memory, Write-back, No-allocate
> * Memory, Write-back, Read-allocate
> * Memory, Write-back, Write-allocate
> * Memory, Write-back, Read and Write-allocate
>
> More info about PMA (section 10.3):
> Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> Example PMA region passed as a DT node from OpenSBI:
> reserved-memory {
> #address-cells = <2>;
> #size-cells = <2>;
> ranges;
>
> pma_resv0 at 58000000 {
> compatible = "shared-dma-pool";
> reg = <0x0 0x58000000 0x0 0x08000000>;
> no-map;
> linux,dma-default;
> };
> };
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> ---
> platform/generic/Kconfig | 8 +
> platform/generic/include/andes45_pma.h | 48 +++
> platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
> platform/generic/renesas/rzfive/objects.mk | 1 +
This file organization does not make sense because after all
these PMA configurations are Andes specific and RZFive
simply using it.
I suggest the following:
1) Rename
platform/generic/include/andes45_pma.h
to
platform/generic/include/andes/andes45_pma.h
2) Rename
platform/generic/renesas/rzfive/andes45-pma.c
to
platform/generic/andes/andes45-pma.c
3) Add platform/generic/andes/Kconfig which defines
"config ANDES45_PMA" but does not depend on
PLATFORM_RENESAS_RZFIVE
4) Include platform/generic/andes/Kconfig in
platform/generic/Kconfig and select ANDES45_PMA
for PLATFORM_RENESAS_RZFIVE
Regards,
Anup
> 4 files changed, 407 insertions(+)
> create mode 100644 platform/generic/include/andes45_pma.h
> create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index c7f198a..9ee9fe3 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -51,3 +51,11 @@ config PLATFORM_STARFIVE_JH7110
> default n
>
> endif
> +
> +if PLATFORM_RENESAS_RZFIVE
> +
> +config ANDES45_PMA
> + bool "Andes PMA support"
> + default n
> +
> +endif
> diff --git a/platform/generic/include/andes45_pma.h b/platform/generic/include/andes45_pma.h
> new file mode 100644
> index 0000000..37ec77c
> --- /dev/null
> +++ b/platform/generic/include/andes45_pma.h
> @@ -0,0 +1,48 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + */
> +
> +#ifndef _ANDES45_PMA_H_
> +#define _ANDES45_PMA_H_
> +
> +#include <sbi/sbi_types.h>
> +
> +#define ANDES45_MAX_PMA_REGIONS 16
> +
> +/* Naturally aligned power of 2 region */
> +#define ANDES45_PMACFG_ETYP_NAPOT 3
> +
> +/* Memory, Non-cacheable, Bufferable */
> +#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF (3 << 2)
> +
> +/**
> + * struct andes45_pma_region - Describes PMA regions
> + *
> + * @pa: Address to be configured in the PMA
> + * @size: Size of the region
> + * @flags: Flags to be set for the PMA region
> + * @dt_populate: Boolean flag indicating if the DT entry should be
> + * populated for the given PMA region
> + * @shared_dma: Boolean flag if set "shared-dma-pool" property will
> + * be set in the DT node
> + * @no_map: Boolean flag if set "no-map" property will be set in the
> + * DT node
> + * @dma_default: Boolean flag if set "linux,dma-default" property will
> + * be set in the DT node. Note Linux expects single node
> + * with this property set.
> + */
> +struct andes45_pma_region {
> + unsigned long pa;
> + unsigned long size;
> + u8 flags:7;
> + bool dt_populate;
> + bool shared_dma;
> + bool no_map;
> + bool dma_default;
> +};
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> + unsigned int pma_regions_count);
> +
> +#endif /* _ANDES45_PMA_H_ */
> diff --git a/platform/generic/renesas/rzfive/andes45-pma.c b/platform/generic/renesas/rzfive/andes45-pma.c
> new file mode 100644
> index 0000000..881a628
> --- /dev/null
> +++ b/platform/generic/renesas/rzfive/andes45-pma.c
> @@ -0,0 +1,350 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + *
> + * Copyright (c) 2020 Andes Technology Corporation
> + *
> + * Authors:
> + * Nick Hu <nickhu@andestech.com>
> + * Nylon Chen <nylon7@andestech.com>
> + * Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> + */
> +
> +#include <andes45_pma.h>
> +#include <libfdt.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +
> +/* Configuration Registers */
> +#define ANDES45_CSR_MMSC_CFG 0xFC2
> +#define ANDES45_CSR_MMSC_PPMA_OFFSET (1 << 30)
> +
> +#define ANDES45_PMAADDR_0 0xBD0
> +
> +#define ANDES45_PMACFG_0 0xBC0
> +
> +static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
> +{
> +#define switchcase_pma_cfg_read(__pma_cfg_off, __val) \
> + case __pma_cfg_off: \
> + __val = csr_read(__pma_cfg_off); \
> + break;
> +#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val) \
> + switchcase_pma_cfg_read(__pma_cfg_off + 0, __val) \
> + switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
> +
> + unsigned long ret = 0;
> +
> + switch (pma_cfg_off) {
> + switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
> +
> + default:
> + sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> + break;
> + }
> +
> + return ret;
> +
> +#undef switchcase_pma_cfg_read_2
> +#undef switchcase_pma_cfg_read
> +}
> +
> +static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
> +{
> +#define switchcase_pma_cfg_write(__pma_cfg_off, __val) \
> + case __pma_cfg_off: \
> + csr_write(__pma_cfg_off, __val); \
> + break;
> +#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val) \
> + switchcase_pma_cfg_write(__pma_cfg_off + 0, __val) \
> + switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
> +
> + switch (pma_cfg_off) {
> + switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
> +
> + default:
> + sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> + break;
> + }
> +
> +#undef switchcase_pma_cfg_write_2
> +#undef switchcase_pma_cfg_write
> +}
> +
> +static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
> +{
> +#define switchcase_pma_write(__pma_addr_off, __val) \
> + case __pma_addr_off: \
> + csr_write(__pma_addr_off, __val); \
> + break;
> +#define switchcase_pma_write_2(__pma_addr_off, __val) \
> + switchcase_pma_write(__pma_addr_off + 0, __val) \
> + switchcase_pma_write(__pma_addr_off + 1, __val)
> +#define switchcase_pma_write_4(__pma_addr_off, __val) \
> + switchcase_pma_write_2(__pma_addr_off + 0, __val) \
> + switchcase_pma_write_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_write_8(__pma_addr_off, __val) \
> + switchcase_pma_write_4(__pma_addr_off + 0, __val) \
> + switchcase_pma_write_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_write_16(__pma_addr_off, __val) \
> + switchcase_pma_write_8(__pma_addr_off + 0, __val) \
> + switchcase_pma_write_8(__pma_addr_off + 8, __val)
> +
> + switch (pma_addr_off) {
> + switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
> +
> + default:
> + sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> + break;
> + }
> +
> +#undef switchcase_pma_write_16
> +#undef switchcase_pma_write_8
> +#undef switchcase_pma_write_4
> +#undef switchcase_pma_write_2
> +#undef switchcase_pma_write
> +}
> +
> +static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
> +{
> +#define switchcase_pma_read(__pma_addr_off, __val) \
> + case __pma_addr_off: \
> + __val = csr_read(__pma_addr_off); \
> + break;
> +#define switchcase_pma_read_2(__pma_addr_off, __val) \
> + switchcase_pma_read(__pma_addr_off + 0, __val) \
> + switchcase_pma_read(__pma_addr_off + 1, __val)
> +#define switchcase_pma_read_4(__pma_addr_off, __val) \
> + switchcase_pma_read_2(__pma_addr_off + 0, __val) \
> + switchcase_pma_read_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_read_8(__pma_addr_off, __val) \
> + switchcase_pma_read_4(__pma_addr_off + 0, __val) \
> + switchcase_pma_read_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_read_16(__pma_addr_off, __val) \
> + switchcase_pma_read_8(__pma_addr_off + 0, __val) \
> + switchcase_pma_read_8(__pma_addr_off + 8, __val)
> +
> + unsigned long ret = 0;
> +
> + switch (pma_addr_off) {
> + switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
> +
> + default:
> + sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> + break;
> + }
> +
> + return ret;
> +
> +#undef switchcase_pma_read_16
> +#undef switchcase_pma_read_8
> +#undef switchcase_pma_read_4
> +#undef switchcase_pma_read_2
> +#undef switchcase_pma_read
> +}
> +
> +static unsigned long
> +andes45_pma_setup(const struct andes45_pma_region *pma_region,
> + unsigned int entry_id)
> +{
> + unsigned long size = pma_region->size;
> + unsigned long addr = pma_region->pa;
> + unsigned int pma_cfg_addr;
> + unsigned long pmacfg_val;
> + unsigned long pmaaddr;
> + char *pmaxcfg;
> +
> + /* Check for 4KiB granularity */
> + if (size < (1 << 12))
> + return SBI_EINVAL;
> +
> + /* Check size is power of 2 */
> + if (size & (size - 1))
> + return SBI_EINVAL;
> +
> + if (entry_id > 15)
> + return SBI_EINVAL;
> +
> + if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
> + return SBI_EINVAL;
> +
> + if ((addr & (size - 1)) != 0)
> + return SBI_EINVAL;
> +
> + pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
> + pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
> + pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
> + *pmaxcfg = 0;
> + *pmaxcfg = pma_region->flags;
> +
> + andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
> +
> + pmaaddr = (addr >> 2) + (size >> 3) - 1;
> +
> + andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
> +
> + return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
> + pmaaddr : SBI_EINVAL;
> +}
> +
> +static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
> + unsigned int index, int parent)
> +{
> + int na = fdt_address_cells(fdt, 0);
> + int ns = fdt_size_cells(fdt, 0);
> + static bool dma_default = false;
> + fdt32_t addr_high, addr_low;
> + fdt32_t size_high, size_low;
> + int subnode, err;
> + fdt32_t reg[4];
> + fdt32_t *val;
> + char name[32];
> +
> + addr_high = (u64)pma->pa >> 32;
> + addr_low = pma->pa;
> + size_high = (u64)pma->size >> 32;
> + size_low = pma->size;
> +
> + if (na > 1 && addr_high)
> + sbi_snprintf(name, sizeof(name),
> + "pma_resv%d@%x,%x", index,
> + addr_high, addr_low);
> + else
> + sbi_snprintf(name, sizeof(name),
> + "pma_resv%d@%x", index,
> + addr_low);
> +
> + subnode = fdt_add_subnode(fdt, parent, name);
> + if (subnode < 0)
> + return subnode;
> +
> + if (pma->shared_dma) {
> + err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
> + if (err < 0)
> + return err;
> + }
> +
> + if (pma->no_map) {
> + err = fdt_setprop_empty(fdt, subnode, "no-map");
> + if (err < 0)
> + return err;
> + }
> +
> + /* Linux allows single linux,dma-default region. */
> + if (pma->dma_default) {
> + if (dma_default)
> + return SBI_EINVAL;
> +
> + err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
> + if (err < 0)
> + return err;
> + dma_default = true;
> + }
> +
> + /* encode the <reg> property value */
> + val = reg;
> + if (na > 1)
> + *val++ = cpu_to_fdt32(addr_high);
> + *val++ = cpu_to_fdt32(addr_low);
> + if (ns > 1)
> + *val++ = cpu_to_fdt32(size_high);
> + *val++ = cpu_to_fdt32(size_low);
> +
> + err = fdt_setprop(fdt, subnode, "reg", reg,
> + (na + ns) * sizeof(fdt32_t));
> + if (err < 0)
> + return err;
> +
> + return 0;
> +}
> +
> +static int andes45_fdt_reserved_memory_fixup(void *fdt,
> + const struct andes45_pma_region *pma,
> + unsigned int entry)
> +{
> + int parent;
> +
> + /* try to locate the reserved memory node */
> + parent = fdt_path_offset(fdt, "/reserved-memory");
> + if (parent < 0) {
> + int na = fdt_address_cells(fdt, 0);
> + int ns = fdt_size_cells(fdt, 0);
> + int err;
> +
> + /* if such node does not exist, create one */
> + parent = fdt_add_subnode(fdt, 0, "reserved-memory");
> + if (parent < 0)
> + return parent;
> +
> + err = fdt_setprop_empty(fdt, parent, "ranges");
> + if (err < 0)
> + return err;
> +
> + err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
> + if (err < 0)
> + return err;
> +
> + err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
> + if (err < 0)
> + return err;
> + }
> +
> + return andes45_fdt_pma_resv(fdt, pma, entry, parent);
> +}
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> + unsigned int pma_regions_count)
> +{
> + unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
> + unsigned int dt_populate_cnt;
> + unsigned int i, j;
> + unsigned long pa;
> + void *fdt;
> + int ret;
> +
> + if (!pma_regions || !pma_regions_count)
> + return 0;
> +
> + if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
> + return SBI_EINVAL;
> +
> + if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
> + return SBI_ENOTSUPP;
> +
> + /* Configure the PMA regions */
> + for (i = 0; i < pma_regions_count; i++) {
> + pa = andes45_pma_setup(&pma_regions[i], i);
> + if (pa == SBI_EINVAL)
> + return SBI_EINVAL;
> + }
> +
> + dt_populate_cnt = 0;
> + for (i = 0; i < pma_regions_count; i++) {
> + if (!pma_regions[i].dt_populate)
> + continue;
> + dt_populate_cnt++;
> + }
> +
> + if (!dt_populate_cnt)
> + return 0;
> +
> + fdt = fdt_get_address();
> +
> + ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
> + if (ret < 0)
> + return ret;
> +
> + for (i = 0, j = 0; i < pma_regions_count; i++) {
> + if (!pma_regions[i].dt_populate)
> + continue;
> +
> + ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> diff --git a/platform/generic/renesas/rzfive/objects.mk b/platform/generic/renesas/rzfive/objects.mk
> index 2e7e37f..fd9e7c4 100644
> --- a/platform/generic/renesas/rzfive/objects.mk
> +++ b/platform/generic/renesas/rzfive/objects.mk
> @@ -6,3 +6,4 @@
>
> carray-platform_override_modules-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas_rzfive
> platform-objs-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas/rzfive/rzfive.o
> +platform-objs-$(CONFIG_ANDES45_PMA) += renesas/rzfive/andes45-pma.o
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata
2023-02-06 0:06 ` [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata Lad Prabhakar
2023-02-07 15:21 ` Yu-Chien Peter Lin
@ 2023-02-13 4:15 ` Anup Patel
2023-02-13 21:48 ` Lad, Prabhakar
1 sibling, 1 reply; 10+ messages in thread
From: Anup Patel @ 2023-02-13 4:15 UTC (permalink / raw)
To: opensbi
On Mon, Feb 6, 2023 at 5:37 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj@bp.renesas.com> wrote:
>
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> errata should be applied to handle cache management.
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> ---
> platform/generic/include/andes/andes45.h | 25 ++++++++++++-
> platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
> 2 files changed, 70 insertions(+), 2 deletions(-)
>
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index 08b3d18..d5d265f 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -4,7 +4,28 @@
> #define CSR_MARCHID_MICROID 0xfff
>
> /* Memory and Miscellaneous Registers */
> -#define CSR_MCACHE_CTL 0x7ca
> -#define CSR_MCCTLCOMMAND 0x7cc
> +#define CSR_MCACHE_CTL 0x7ca
> +#define CSR_MCCTLCOMMAND 0x7cc
> +#define CSR_MICM_CFG 0xfc0
> +#define CSR_MDCM_CFG 0xfc1
> +#define CSR_MMSC_CFG 0xfc2
> +
> +#define MISA_20_OFFSET 20
> +#define MISA_20_MASK (0x1 << MISA_20_OFFSET)
> +
> +#define MICM_CFG_ISZ_OFFSET 6
> +#define MICM_CFG_ISZ_MASK (0x7 << MICM_CFG_ISZ_OFFSET)
> +
> +#define MDCM_CFG_DSZ_OFFSET 6
> +#define MDCM_CFG_DSZ_MASK (0x7 << MDCM_CFG_DSZ_OFFSET)
> +
> +#define MMSC_CFG_CCTLCSR_OFFSET 16
> +#define MMSC_CFG_CCTLCSR_MASK (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> +#define MMSC_IOCP_OFFSET 47
> +#define MMSC_IOCP_MASK (0x1ULL << MMSC_IOCP_OFFSET)
> +
> +#define MCACHE_CTL_CCTL_SUEN_OFFSET 8
> +#define MCACHE_CTL_CCTL_SUEN_MASK (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> +
>
> #endif /* _RISCV_ANDES45_H */
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index eee9c51..552c747 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -4,11 +4,17 @@
> *
> */
>
> +#include <andes/andes45.h>
> #include <andes45_pma.h>
> #include <platform_override.h>
> #include <sbi/sbi_domain.h>
> +#include <sbi/sbi_error.h>
> #include <sbi_utils/fdt/fdt_helper.h>
>
> +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND 0
> +
> +#define ANDESTECH_SBI_EXT_ANDES 0x900031e
> +
> static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
> {
> .pa = 0x58000000,
> @@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
> array_size(renesas_rzfive_pma_regions));
> }
>
> +static bool renesas_rzfive_cpu_cache_controlable(void)
> +{
> + return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
> + (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
> + (csr_read(CSR_MISA) & MISA_20_MASK) &&
> + (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
> + (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
> +}
> +
> +static bool renesas_rzfive_cpu_iocp_disabled(void)
> +{
> + return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
> +}
> +
> +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
> +{
> + return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
> +}
> +
> +static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
> + const struct sbi_trap_regs *regs,
> + unsigned long *out_value,
> + struct sbi_trap_info *out_trap,
> + const struct fdt_match *match)
> +{
> + if (extid != ANDESTECH_SBI_EXT_ANDES)
> + return SBI_EINVAL;
This is also looking strange because you are checking Andes vendor
extension in RZ Five sources.
I will send a small patch which will help you simplify this over here. You
can include that patch in this series in the next revision.
Regards,
Anup
> +
> + switch (funcid) {
> + case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
> + *out_value = renesas_rzfive_apply_iocp_sw_workaround();
> + break;
> +
> + default:
> + break;
> + }
> +
> + return 0;
> +}
> +
> int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
> {
> /*
> @@ -55,4 +101,5 @@ const struct platform_override renesas_rzfive = {
> .match_table = renesas_rzfive_match,
> .early_init = renesas_rzfive_early_init,
> .final_init = renesas_rzfive_final_init,
> + .vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
> };
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA
2023-02-13 4:07 ` Anup Patel
@ 2023-02-13 21:46 ` Lad, Prabhakar
0 siblings, 0 replies; 10+ messages in thread
From: Lad, Prabhakar @ 2023-02-13 21:46 UTC (permalink / raw)
To: opensbi
Hi Anup,
Thank you for the review.
On Mon, Feb 13, 2023 at 4:07 AM Anup Patel <anup@brainfault.org> wrote:
>
> On Mon, Feb 6, 2023 at 5:36 AM Lad Prabhakar
> <prabhakar.mahadev-lad.rj@bp.renesas.com> wrote:
> >
> > I/O Coherence Port (IOCP) provides an AXI interface for connecting
> > external non-caching masters, such as DMA controllers. The accesses
> > from IOCP are coherent with D-Caches and L2 Cache.
> >
> > IOCP is a specification option and is disabled on the Renesas RZ/Five
> > SoC due to this reason IP blocks using DMA will fail.
> >
> > The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> > block that allows dynamic adjustment of memory attributes in the runtime.
> > It contains a configurable amount of PMA entries implemented as CSR
> > registers to control the attributes of memory locations in interest.
> > Below are the memory attributes supported:
> > * Device, Non-bufferable
> > * Device, bufferable
> > * Memory, Non-cacheable, Non-bufferable
<snip>
> > platform/generic/Kconfig | 8 +
> > platform/generic/include/andes45_pma.h | 48 +++
> > platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
> > platform/generic/renesas/rzfive/objects.mk | 1 +
>
> This file organization does not make sense because after all
> these PMA configurations are Andes specific and RZFive
> simply using it.
>
> I suggest the following:
> 1) Rename
> platform/generic/include/andes45_pma.h
> to
> platform/generic/include/andes/andes45_pma.h
> 2) Rename
> platform/generic/renesas/rzfive/andes45-pma.c
> to
> platform/generic/andes/andes45-pma.c
> 3) Add platform/generic/andes/Kconfig which defines
> "config ANDES45_PMA" but does not depend on
> PLATFORM_RENESAS_RZFIVE
> 4) Include platform/generic/andes/Kconfig in
> platform/generic/Kconfig and select ANDES45_PMA
> for PLATFORM_RENESAS_RZFIVE
>
Ok, I'll do the above in the next version.
Cheers,
Prabhakar
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata
2023-02-13 4:15 ` Anup Patel
@ 2023-02-13 21:48 ` Lad, Prabhakar
0 siblings, 0 replies; 10+ messages in thread
From: Lad, Prabhakar @ 2023-02-13 21:48 UTC (permalink / raw)
To: opensbi
Hi Anup,
Thank you for the review.
On Mon, Feb 13, 2023 at 4:15 AM Anup Patel <anup@brainfault.org> wrote:
>
> On Mon, Feb 6, 2023 at 5:37 AM Lad Prabhakar
> <prabhakar.mahadev-lad.rj@bp.renesas.com> wrote:
> >
> > I/O Coherence Port (IOCP) provides an AXI interface for connecting
> > external non-caching masters, such as DMA controllers. The accesses
> > from IOCP are coherent with D-Caches and L2 Cache.
> >
> > IOCP is a specification option and is disabled on the Renesas RZ/Five
> > SoC due to this reason IP blocks using DMA will fail.
> >
> > As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> > software. Firstly OpenSBI configures the memory region as
> > "Memory, Non-cacheable, Bufferable" and passes this region as a global
> > shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> > allocations happen from this region and synchronization callbacks are
> > implemented to synchronize when doing DMA transactions.
> >
> > RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> > errata should be applied to handle cache management.
> >
> > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > ---
> > platform/generic/include/andes/andes45.h | 25 ++++++++++++-
> > platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
> > 2 files changed, 70 insertions(+), 2 deletions(-)
> >
> > diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> > index 08b3d18..d5d265f 100644
> > --- a/platform/generic/include/andes/andes45.h
> > +++ b/platform/generic/include/andes/andes45.h
> > @@ -4,7 +4,28 @@
> > #define CSR_MARCHID_MICROID 0xfff
> >
> > /* Memory and Miscellaneous Registers */
> > -#define CSR_MCACHE_CTL 0x7ca
> > -#define CSR_MCCTLCOMMAND 0x7cc
> > +#define CSR_MCACHE_CTL 0x7ca
> > +#define CSR_MCCTLCOMMAND 0x7cc
> > +#define CSR_MICM_CFG 0xfc0
> > +#define CSR_MDCM_CFG 0xfc1
> > +#define CSR_MMSC_CFG 0xfc2
> > +
> > +#define MISA_20_OFFSET 20
> > +#define MISA_20_MASK (0x1 << MISA_20_OFFSET)
> > +
> > +#define MICM_CFG_ISZ_OFFSET 6
> > +#define MICM_CFG_ISZ_MASK (0x7 << MICM_CFG_ISZ_OFFSET)
> > +
> > +#define MDCM_CFG_DSZ_OFFSET 6
> > +#define MDCM_CFG_DSZ_MASK (0x7 << MDCM_CFG_DSZ_OFFSET)
> > +
> > +#define MMSC_CFG_CCTLCSR_OFFSET 16
> > +#define MMSC_CFG_CCTLCSR_MASK (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> > +#define MMSC_IOCP_OFFSET 47
> > +#define MMSC_IOCP_MASK (0x1ULL << MMSC_IOCP_OFFSET)
> > +
> > +#define MCACHE_CTL_CCTL_SUEN_OFFSET 8
> > +#define MCACHE_CTL_CCTL_SUEN_MASK (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> > +
> >
> > #endif /* _RISCV_ANDES45_H */
> > diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> > index eee9c51..552c747 100644
> > --- a/platform/generic/renesas/rzfive/rzfive.c
> > +++ b/platform/generic/renesas/rzfive/rzfive.c
> > @@ -4,11 +4,17 @@
> > *
> > */
> >
> > +#include <andes/andes45.h>
> > #include <andes45_pma.h>
> > #include <platform_override.h>
> > #include <sbi/sbi_domain.h>
> > +#include <sbi/sbi_error.h>
> > #include <sbi_utils/fdt/fdt_helper.h>
> >
> > +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND 0
> > +
> > +#define ANDESTECH_SBI_EXT_ANDES 0x900031e
> > +
> > static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
> > {
> > .pa = 0x58000000,
> > @@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
> > array_size(renesas_rzfive_pma_regions));
> > }
> >
> > +static bool renesas_rzfive_cpu_cache_controlable(void)
> > +{
> > + return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
> > + (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
> > + (csr_read(CSR_MISA) & MISA_20_MASK) &&
> > + (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
> > + (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
> > +}
> > +
> > +static bool renesas_rzfive_cpu_iocp_disabled(void)
> > +{
> > + return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
> > +}
> > +
> > +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
> > +{
> > + return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
> > +}
> > +
> > +static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
> > + const struct sbi_trap_regs *regs,
> > + unsigned long *out_value,
> > + struct sbi_trap_info *out_trap,
> > + const struct fdt_match *match)
> > +{
> > + if (extid != ANDESTECH_SBI_EXT_ANDES)
> > + return SBI_EINVAL;
>
> This is also looking strange because you are checking Andes vendor
> extension in RZ Five sources.
>
> I will send a small patch which will help you simplify this over here. You
> can include that patch in this series in the next revision.
>
OK.
Cheers,
Prabhakar
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-02-13 21:48 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-02-06 0:06 [PATCH v4 0/3] Renesas RZ/Five add support to configure PMA and add vendor specific SBI extension Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA Lad Prabhakar
2023-02-07 15:08 ` Yu-Chien Peter Lin
2023-02-13 4:07 ` Anup Patel
2023-02-13 21:46 ` Lad, Prabhakar
2023-02-06 0:06 ` [PATCH v4 2/3] platform: generic: renesas: rzfive: Configure the PMA region Lad Prabhakar
2023-02-06 0:06 ` [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata Lad Prabhakar
2023-02-07 15:21 ` Yu-Chien Peter Lin
2023-02-13 4:15 ` Anup Patel
2023-02-13 21:48 ` Lad, Prabhakar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox