[PATCH 00/13] OpenSBI RISC-V AIA Support

[PATCH 00/13] OpenSBI RISC-V AIA Support

[Anup Patel]

The advanced interrupt architecture (AIA) extends the per-HART local
interrupt support. Along with this, it also adds IMSIC (MSI contrllor)
and Advanced PLIC (wired interrupt controller).

The latest AIA draft specification can be found here:
https://github.com/riscv/riscv-aia/releases/download/0.2-draft.28/riscv-interrupts-028.pdf

To test series, we require:
1) QEMU from riscv_aia_v6 branch of https://github.com/avpatel/qemu.git
2) Linux from riscv_aia_v1 branch of https://github.com/avpatel/linux.git

This series can be found riscv_aia_v1 branch at:
https://github.com/avpatel/opensbi.git

Anup Patel (13):
  include: sbi: Add AIA related CSR defines
  lib: sbi: Detect AIA CSRs at boot-time
  lib: sbi: Use AIA CSRs for local interrupts when available
  lib: sbi: Add sbi_trap_set_external_irqfn() API
  lib: utils/irqchip: Allow multiple FDT irqchip drivers
  include: sbi: Introduce nascent_init() platform callback
  lib: sbi: Enable mie.MEIE bit for IPIs based on external interrupts.
  lib: utils/irqchip: Add IMSIC library
  lib: utils/irqchip: Add FDT based driver for IMSIC
  lib: utils: Disable appropriate IMSIC DT nodes in fdt_fixups()
  lib: utils/irqchip: Add APLIC initialization library
  lib: utils/irqchip: Add FDT based driver for APLIC
  lib: utils: Disable appropriate APLIC DT nodes in fdt_fixups()

 include/sbi/riscv_encoding.h          |  76 +++++++
 include/sbi/sbi_hart.h                |   4 +-
 include/sbi/sbi_platform.h            |  17 ++
 include/sbi/sbi_trap.h                |   2 +
 include/sbi_utils/fdt/fdt_fixup.h     |  29 ++-
 include/sbi_utils/fdt/fdt_helper.h    |  10 +
 include/sbi_utils/irqchip/aplic.h     |  47 +++++
 include/sbi_utils/irqchip/imsic.h     |  50 +++++
 lib/sbi/sbi_hart.c                    |  23 +++
 lib/sbi/sbi_hsm.c                     |   4 +-
 lib/sbi/sbi_init.c                    |  17 +-
 lib/sbi/sbi_trap.c                    |  86 +++++++-
 lib/utils/fdt/fdt_fixup.c             |  43 +++-
 lib/utils/fdt/fdt_helper.c            | 262 +++++++++++++++++++++++
 lib/utils/irqchip/aplic.c             | 279 +++++++++++++++++++++++++
 lib/utils/irqchip/fdt_irqchip.c       |  41 +++-
 lib/utils/irqchip/fdt_irqchip_aplic.c |  56 +++++
 lib/utils/irqchip/fdt_irqchip_imsic.c | 106 ++++++++++
 lib/utils/irqchip/imsic.c             | 287 ++++++++++++++++++++++++++
 lib/utils/irqchip/objects.mk          |   4 +
 platform/generic/platform.c           |  12 ++
 21 files changed, 1427 insertions(+), 28 deletions(-)
 create mode 100644 include/sbi_utils/irqchip/aplic.h
 create mode 100644 include/sbi_utils/irqchip/imsic.h
 create mode 100644 lib/utils/irqchip/aplic.c
 create mode 100644 lib/utils/irqchip/fdt_irqchip_aplic.c
 create mode 100644 lib/utils/irqchip/fdt_irqchip_imsic.c
 create mode 100644 lib/utils/irqchip/imsic.c

-- 
2.25.1

[PATCH 01/13] include: sbi: Add AIA related CSR defines

[Anup Patel]

The RISC-V AIA specification improves handling of per-HART local
interrupts in a backward compatible manner. This patch adds defines
for the new RISC-V AIA CSRs.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 include/sbi/riscv_encoding.h | 76 ++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
index 8a29f8e..03f85cb 100644
--- a/include/sbi/riscv_encoding.h
+++ b/include/sbi/riscv_encoding.h
@@ -173,6 +173,10 @@
 #define HGATP_MODE_SHIFT		HGATP32_MODE_SHIFT
 #endif
 
+#define TOPI_IID_SHIFT			16
+#define TOPI_IID_MASK			0xfff
+#define TOPI_IPRIO_MASK		0xff
+
 #if __riscv_xlen == 64
 #define MHPMEVENT_OF			(_UL(1) << 63)
 #define MHPMEVENT_MINH			(_UL(1) << 62)
@@ -303,6 +307,24 @@
 /* Supervisor Protection and Translation */
 #define CSR_SATP			0x180
 
+/* Supervisor-Level Window to Indirectly Accessed Registers (AIA) */
+#define CSR_SISELECT			0x150
+#define CSR_SIREG			0x151
+
+/* Supervisor-Level Interrupts (AIA) */
+#define CSR_STOPI			0xdb0
+
+/* Supervisor-Level IMSIC Interface (AIA) */
+#define CSR_SSETEIPNUM			0x158
+#define CSR_SCLREIPNUM			0x159
+#define CSR_SSETEIENUM			0x15a
+#define CSR_SCLREIENUM			0x15b
+#define CSR_STOPEI			0x15c
+
+/* Supervisor-Level High-Half CSRs (AIA) */
+#define CSR_SIEH			0x114
+#define CSR_SIPH			0x154
+
 /* ===== Hypervisor-level CSRs ===== */
 
 /* Hypervisor Trap Setup (H-extension) */
@@ -338,6 +360,35 @@
 #define CSR_VSIP			0x244
 #define CSR_VSATP			0x280
 
+/* Virtual Interrupts and Interrupt Priorities (H-extension with AIA) */
+#define CSR_HVIEN			0x608
+#define CSR_HVICTL			0x609
+#define CSR_HVIPRIO1			0x646
+#define CSR_HVIPRIO2			0x647
+
+/* VS-Level Window to Indirectly Accessed Registers (H-extension with AIA) */
+#define CSR_VSISELECT			0x250
+#define CSR_VSIREG			0x251
+
+/* VS-Level Interrupts (H-extension with AIA) */
+#define CSR_VSTOPI			0xeb0
+
+/* VS-Level IMSIC Interface (H-extension with AIA) */
+#define CSR_VSSETEIPNUM		0x258
+#define CSR_VSCLREIPNUM		0x259
+#define CSR_VSSETEIENUM		0x25a
+#define CSR_VSCLREIENUM		0x25b
+#define CSR_VSTOPEI			0x25c
+
+/* Hypervisor and VS-Level High-Half CSRs (H-extension with AIA) */
+#define CSR_HIDELEGH			0x613
+#define CSR_HVIENH			0x618
+#define CSR_HVIPH			0x655
+#define CSR_HVIPRIO1H			0x656
+#define CSR_HVIPRIO2H			0x657
+#define CSR_VSIEH			0x214
+#define CSR_VSIPH			0x254
+
 /* ===== Machine-level CSRs ===== */
 
 /* Machine Information Registers */
@@ -589,6 +640,31 @@
 #define CSR_DSCRATCH0			0x7b2
 #define CSR_DSCRATCH1			0x7b3
 
+/* Machine-Level Window to Indirectly Accessed Registers (AIA) */
+#define CSR_MISELECT			0x350
+#define CSR_MIREG			0x351
+
+/* Machine-Level Interrupts (AIA) */
+#define CSR_MTOPI			0xfb0
+
+/* Machine-Level IMSIC Interface (AIA) */
+#define CSR_MSETEIPNUM			0x358
+#define CSR_MCLREIPNUM			0x359
+#define CSR_MSETEIENUM			0x35a
+#define CSR_MCLREIENUM			0x35b
+#define CSR_MTOPEI			0x35c
+
+/* Virtual Interrupts for Supervisor Level (AIA) */
+#define CSR_MVIEN			0x308
+#define CSR_MVIP			0x309
+
+/* Machine-Level High-Half CSRs (AIA) */
+#define CSR_MIDELEGH			0x313
+#define CSR_MIEH			0x314
+#define CSR_MVIENH			0x318
+#define CSR_MVIPH			0x319
+#define CSR_MIPH			0x354
+
 /* ===== Trap/Exception Causes ===== */
 
 #define CAUSE_MISALIGNED_FETCH		0x0
-- 
2.25.1

[PATCH 02/13] lib: sbi: Detect AIA CSRs at boot-time

[Anup Patel]

We extend HART feature detection to discover AIA CSRs at boot-time.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 include/sbi/sbi_hart.h |  4 +++-
 lib/sbi/sbi_hart.c     | 23 +++++++++++++++++++++++
 2 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
index d2db9d6..a83b45b 100644
--- a/include/sbi/sbi_hart.h
+++ b/include/sbi/sbi_hart.h
@@ -24,9 +24,11 @@ enum sbi_hart_features {
 	SBI_HART_HAS_SSCOFPMF = (1 << 3),
 	/** HART has timer csr implementation in hardware */
 	SBI_HART_HAS_TIME = (1 << 4),
+	/** HART has AIA local interrupt CSRs */
+	SBI_HART_HAS_AIA = (1 << 5),
 
 	/** Last index of Hart features*/
-	SBI_HART_HAS_LAST_FEATURE = SBI_HART_HAS_TIME,
+	SBI_HART_HAS_LAST_FEATURE = SBI_HART_HAS_AIA,
 };
 
 struct sbi_scratch;
diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index d9a15d9..c16b395 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -283,6 +283,9 @@ static inline char *sbi_hart_feature_id2string(unsigned long feature)
 	case SBI_HART_HAS_TIME:
 		fstr = "time";
 		break;
+	case SBI_HART_HAS_AIA:
+		fstr = "aia";
+		break;
 	default:
 		break;
 	}
@@ -506,6 +509,26 @@ __mhpm_skip:
 	csr_read_allowed(CSR_TIME, (unsigned long)&trap);
 	if (!trap.cause)
 		hfeatures->features |= SBI_HART_HAS_TIME;
+
+	/* Detect if hart has AIA local interrupt CSRs */
+	csr_read_allowed(CSR_MTOPI, (unsigned long)&trap);
+	if (trap.cause)
+		goto __aia_skip;
+	csr_read_allowed(CSR_MISELECT, (unsigned long)&trap);
+	if (trap.cause)
+		goto __aia_skip;
+	csr_write_allowed(CSR_MISELECT, (unsigned long)&trap, 0x30);
+	if (trap.cause)
+		goto __aia_skip;
+	val = csr_read_allowed(CSR_MIREG, (unsigned long)&trap);
+	if (trap.cause)
+		goto __aia_skip;
+	csr_write_allowed(CSR_MIREG, (unsigned long)&trap, val);
+	if (trap.cause)
+		goto __aia_skip;
+	hfeatures->features |= SBI_HART_HAS_AIA;
+__aia_skip:
+	return;
 }
 
 int sbi_hart_reinit(struct sbi_scratch *scratch)
-- 
2.25.1

[PATCH 02/13] lib: sbi: Detect AIA CSRs at boot-time

[Anup Patel]

On Fri, Jan 28, 2022 at 1:17 PM Atish Patra <atishp@atishpatra.org> wrote:
>
>
>
> On Tue, Jan 4, 2022 at 2:10 AM Anup Patel <apatel@ventanamicro.com> wrote:
>>
>> We extend HART feature detection to discover AIA CSRs at boot-time.
>>
>> Signed-off-by: Anup Patel <anup.patel@wdc.com>
>> Signed-off-by: Anup Patel <apatel@ventanamicro.com>
>> ---
>>  include/sbi/sbi_hart.h |  4 +++-
>>  lib/sbi/sbi_hart.c     | 23 +++++++++++++++++++++++
>>  2 files changed, 26 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
>> index d2db9d6..a83b45b 100644
>> --- a/include/sbi/sbi_hart.h
>> +++ b/include/sbi/sbi_hart.h
>> @@ -24,9 +24,11 @@ enum sbi_hart_features {
>>         SBI_HART_HAS_SSCOFPMF = (1 << 3),
>>         /** HART has timer csr implementation in hardware */
>>         SBI_HART_HAS_TIME = (1 << 4),
>> +       /** HART has AIA local interrupt CSRs */
>> +       SBI_HART_HAS_AIA = (1 << 5),
>>
>>         /** Last index of Hart features*/
>> -       SBI_HART_HAS_LAST_FEATURE = SBI_HART_HAS_TIME,
>> +       SBI_HART_HAS_LAST_FEATURE = SBI_HART_HAS_AIA,
>>  };
>>
>>  struct sbi_scratch;
>> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
>> index d9a15d9..c16b395 100644
>> --- a/lib/sbi/sbi_hart.c
>> +++ b/lib/sbi/sbi_hart.c
>> @@ -283,6 +283,9 @@ static inline char *sbi_hart_feature_id2string(unsigned long feature)
>>         case SBI_HART_HAS_TIME:
>>                 fstr = "time";
>>                 break;
>> +       case SBI_HART_HAS_AIA:
>> +               fstr = "aia";
>> +               break;
>>         default:
>>                 break;
>>         }
>> @@ -506,6 +509,26 @@ __mhpm_skip:
>>         csr_read_allowed(CSR_TIME, (unsigned long)&trap);
>>         if (!trap.cause)
>>                 hfeatures->features |= SBI_HART_HAS_TIME;
>> +
>> +       /* Detect if hart has AIA local interrupt CSRs */
>> +       csr_read_allowed(CSR_MTOPI, (unsigned long)&trap);
>> +       if (trap.cause)
>> +               goto __aia_skip;
>> +       csr_read_allowed(CSR_MISELECT, (unsigned long)&trap);
>> +       if (trap.cause)
>> +               goto __aia_skip;
>> +       csr_write_allowed(CSR_MISELECT, (unsigned long)&trap, 0x30);
>
>
> You can use __check_csr directly here as you are not using the value returned from csr_read_allowed.
> Some comment for the magic value  0x30 is necessary as well.

We don't __check_csr() here because some CSRs (such as MTOPI) are read-only.

>
> I have a slightly orthogonal question as well. AFAIK, all these CSRs are mandatory for AIA.
> Do we need to detect all the CSRs every time? Absence of 1 CSR is enough to indicate that AIA is not
> present. I guess you want all the CSR to cover the buggy hardware use case.
>
> However, that is a corner case which can be handled in a platform specific manner. I am just worried
> that this may blow up in the future which would hurt the boot time in the long run.

You are right. Checking all these CSRs for AIA presence is an overkill.

We can simply check read-only MTOPI CSR and be done with it. I will
update this in the next patch revision.

Regards,
Anup

>
>> +       if (trap.cause)
>> +               goto __aia_skip;
>> +       val = csr_read_allowed(CSR_MIREG, (unsigned long)&trap);
>> +       if (trap.cause)
>> +               goto __aia_skip;
>> +       csr_write_allowed(CSR_MIREG, (unsigned long)&trap, val);
>> +       if (trap.cause)
>> +               goto __aia_skip;
>> +       hfeatures->features |= SBI_HART_HAS_AIA;
>> +__aia_skip:
>> +       return;
>>  }
>>
>>  int sbi_hart_reinit(struct sbi_scratch *scratch)
>> --
>> 2.25.1
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
>
>
>
> --
> Regards,
> Atish