[PATCH 0/3] Support OpenSBI firmware header

[PATCH 0/3] Support OpenSBI firmware header

[Zong Li]

This series adds OpenSBI firmware header. The motivation is when a
system where the previous booting stage runs on a dedicated boot
processor, it can load and patch the OpenSBI image in DRAM and then
release the application hart that runs OpenSBI, but it can never execute
on that hart and therefore cannot place anything in its registers, such
as $a0, $a1 and $a2

Every OpenSBI firmware image, regardless of its type (FW_DYNAMIC,
FW_JUMP or FW_PAYLOAD), starts with a fixed 128-byte header. The header
serves two purposes:

1. it lets the previous booting stage recognize an OpenSBI firmware
   image and find out what it was built for, and
2. it gives the previous booting stage a well-known place inside the
   image to hand over the boot arguments by patching a few words,
   instead of setting up the *a0*, *a1* and *a2* registers.

This series also provides a document 'docs/firmware/fw_header.md' to
describe the layout of the 128-byte OpenSBI firmware header, the
a0/a1/a2 override flags, what a previous booting stage is expected to put
in each register, and an example of how to patch the header.

Thanks Anup for discussing it and providing the ideas

Zong Li (3):
  firmware: fw_base.S: add a formatted OpenSBI firmware header
  firmware: fw_base.S: override a0, a1 and a2 from the firmware header
  docs: firmware: document the OpenSBI firmware header

 docs/firmware/fw.md        |   9 +++
 docs/firmware/fw_header.md | 127 +++++++++++++++++++++++++++++++++++++
 firmware/fw_base.S         | 100 +++++++++++++++++++++++++++++
 3 files changed, 236 insertions(+)
 create mode 100644 docs/firmware/fw_header.md

-- 
2.43.7


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[PATCH 1/3] firmware: fw_base.S: add a formatted OpenSBI firmware header

[Zong Li]

Add a fixed 128-byte header at the very beginning of every OpenSBI
firmware image, regardless of the firmware type. The header lets the
previous booting stage identify an OpenSBI image and, more importantly,
gives it a well-known place inside the image to hand over information
to OpenSBI by patching a few words instead of setting up registers.

This first patch only introduces the layout:

  - a 4-byte jump over the header to _start_real, so that _start stays
    the entry point of the image. The jump is assembled with
    '.option norvc' so that the fields behind it are always at a fixed
    offset, even for a build with compressed instructions enabled,

  - the 'OSBI' magic, a header version, the XLEN the firmware was built
    for and the size of the firmware image, so that the previous booting
    stage can validate the image and figure out how much memory it
    occupies,

  - a flags word and three 8-byte override values, which are unused
    (and zero) for now and are wired up by the next patch.

The layout is deliberately identical for RV32 and RV64 so that the
previous booting stage can parse the header without knowing the XLEN of
the firmware upfront. Each override value is followed by a reserved
8-byte slot so that the same layout can be extended to RV128 later.

Suggested-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Zong Li <zong.li@sifive.com>
---
 firmware/fw_base.S | 68 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/firmware/fw_base.S b/firmware/fw_base.S
index 0c5c65c1..dd2adb8a 100644
--- a/firmware/fw_base.S
+++ b/firmware/fw_base.S
@@ -17,6 +17,15 @@
 #define BOOT_LOTTERY_ACQUIRED		1
 #define BOOT_STATUS_BOOT_HART_DONE	1
 
+/* OpenSBI firmware header */
+#define FW_HEADER_MAGIC_VALUE		0x4942534f /* ASCII string "OSBI" */
+#define FW_HEADER_VERSION		0x1
+#define FW_HEADER_SIZE			128
+#define FW_HEADER_RESERVED_OFFSET	0x48
+#define FW_HEADER_FLAGS_OVERRIDE_A0	(1 << 0)
+#define FW_HEADER_FLAGS_OVERRIDE_A1	(1 << 1)
+#define FW_HEADER_FLAGS_OVERRIDE_A2	(1 << 2)
+
 .macro	MOV_3R __d0, __s0, __d1, __s1, __d2, __s2
 	add	\__d0, \__s0, zero
 	add	\__d1, \__s1, zero
@@ -44,8 +53,67 @@
 	.section .entry, "ax", %progbits
 	.align 3
 	.globl _start
+	.globl _start_real
 	.globl _start_warm
 _start:
+	/*
+	 * OpenSBI firmware header
+	 *
+	 * Every OpenSBI firmware image starts with this fixed 128-byte
+	 * header. The layout is identical for RV32 and RV64 (and can be
+	 * extended for RV128 by using the reserved half of each override
+	 * field), so the previous booting stage can parse the header without
+	 * knowing the XLEN of the firmware upfront.
+	 *
+	 * Offset  Size  Field
+	 *   0x00     4  jump to _start_real
+	 *   0x04     4  magic ('OSBI')
+	 *   0x08     4  header version
+	 *   0x0c     4  firmware XLEN
+	 *   0x10     4  firmware size (_fw_end - _fw_start)
+	 *   0x14     4  flags (FW_HEADER_FLAGS_*)
+	 *   0x18     8  override value for a0
+	 *   0x20     8  reserved (upper half of the a0 value on RV128)
+	 *   0x28     8  override value for a1
+	 *   0x30     8  reserved (upper half of the a1 value on RV128)
+	 *   0x38     8  override value for a2
+	 *   0x40     8  reserved (upper half of the a2 value on RV128)
+	 *   0x48    56  reserved
+	 */
+_fw_header_jump:
+	/*
+	 * Must stay a 4-byte instruction so that the fields below are always
+	 * at a fixed offset. 'norvc' keeps the assembler from picking c.j and
+	 * 'norelax' keeps the linker from compressing it later on, which it
+	 * would otherwise happily do for a jump this short.
+	 */
+	.option push
+	.option norvc
+	.option norelax
+	j	_start_real
+	.option pop
+_fw_header_magic:
+	.word	FW_HEADER_MAGIC_VALUE
+_fw_header_version:
+	.word	FW_HEADER_VERSION
+_fw_header_xlen:
+	.word	__riscv_xlen
+_fw_header_size:
+	.word	(_fw_end - _fw_start)
+_fw_header_flags:
+	.word	0
+_fw_header_override_a0:
+	.dword	0
+	.dword	0			/* reserved */
+_fw_header_override_a1:
+	.dword	0
+	.dword	0			/* reserved */
+_fw_header_override_a2:
+	.dword	0
+	.dword	0			/* reserved */
+	/* Reserved, pads the header up to FW_HEADER_SIZE bytes */
+	.fill	(FW_HEADER_SIZE - FW_HEADER_RESERVED_OFFSET), 1, 0
+_start_real:
 	/* Find preferred boot HART id */
 	MOV_3R	s0, a0, s1, a1, s2, a2
 	call	fw_boot_hart
-- 
2.43.7


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[PATCH 2/3] firmware: fw_base.S: override a0, a1 and a2 from the firmware header

[Zong Li]

Wire up the flags word and the three override values in the OpenSBI
firmware header. When the previous booting stage sets one of the
FW_HEADER_FLAGS_OVERRIDE_A[012] bits, the cold boot path replaces the
matching register with the value stored in the header before doing
anything else with the boot arguments.

This lets a previous booting stage which cannot pass the boot arguments
in registers hand them over by patching a few words in the firmware
image instead. For example, a booting stage running on a dedicated boot
processor can load the OpenSBI image, patch the header with the hart id,
the DTB address and, for FW_DYNAMIC, the address of a struct
fw_dynamic_info it built somewhere in DRAM, and then release the hart
that runs OpenSBI, without ever being able to set up that hart's
registers itself.

The flags word is zero in a freshly built image, so nothing is
overridden and every existing booting stage keeps passing a0, a1 and a2
in registers as before.

Suggested-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Zong Li <zong.li@sifive.com>
---
 firmware/fw_base.S | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/firmware/fw_base.S b/firmware/fw_base.S
index dd2adb8a..bf458ba0 100644
--- a/firmware/fw_base.S
+++ b/firmware/fw_base.S
@@ -114,6 +114,38 @@ _fw_header_override_a2:
 	/* Reserved, pads the header up to FW_HEADER_SIZE bytes */
 	.fill	(FW_HEADER_SIZE - FW_HEADER_RESERVED_OFFSET), 1, 0
 _start_real:
+	/*
+	 * Override a0, a1 and a2 with the values from the firmware header.
+	 *
+	 * This is for a previous booting stage which cannot pass the boot
+	 * arguments in registers, but can patch the firmware image before
+	 * jumping to it. Nothing is overridden when the previous booting
+	 * stage left the flags word alone, so the registers passed by all
+	 * existing booting stages are used as-is.
+	 *
+	 * This runs before relocation, so only PC-relative addressing is
+	 * used. t0, t1 and t2 are free to use here: the boot arguments live
+	 * in a0-a4 and every other register is reset further down the cold
+	 * boot path.
+	 */
+	lla	t0, _fw_header_flags
+	lw	t0, (t0)
+	andi	t1, t0, FW_HEADER_FLAGS_OVERRIDE_A0
+	beqz	t1, _skip_override_a0
+	lla	t2, _fw_header_override_a0
+	REG_L	a0, (t2)
+_skip_override_a0:
+	andi	t1, t0, FW_HEADER_FLAGS_OVERRIDE_A1
+	beqz	t1, _skip_override_a1
+	lla	t2, _fw_header_override_a1
+	REG_L	a1, (t2)
+_skip_override_a1:
+	andi	t1, t0, FW_HEADER_FLAGS_OVERRIDE_A2
+	beqz	t1, _skip_override_a2
+	lla	t2, _fw_header_override_a2
+	REG_L	a2, (t2)
+_skip_override_a2:
+
 	/* Find preferred boot HART id */
 	MOV_3R	s0, a0, s1, a1, s2, a2
 	call	fw_boot_hart
-- 
2.43.7


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[PATCH 3/3] docs: firmware: document the OpenSBI firmware header

[Zong Li]

Add docs/firmware/fw_header.md describing the layout of the 128-byte
OpenSBI firmware header, the a0/a1/a2 override flags, what a previous
booting stage is expected to put in each register, and an example of how
to patch the header. Also point at it from fw.md, next to the
description of the registers the previous booting stage passes.

Signed-off-by: Zong Li <zong.li@sifive.com>
---
 docs/firmware/fw.md        |   9 +++
 docs/firmware/fw_header.md | 127 +++++++++++++++++++++++++++++++++++++
 2 files changed, 136 insertions(+)
 create mode 100644 docs/firmware/fw_header.md

diff --git a/docs/firmware/fw.md b/docs/firmware/fw.md
index d2980967..6b5ff153 100644
--- a/docs/firmware/fw.md
+++ b/docs/firmware/fw.md
@@ -16,6 +16,11 @@ of RISC-V CPU:
 * device tree blob address in memory via *a1* register. The address must
   be aligned to 8 bytes.
 
+A previous booting stage which cannot set up these registers can instead pass
+the same information by patching the OpenSBI firmware header, which is present
+at the beginning of every OpenSBI firmware image. See *[FW_HEADER]* for
+details.
+
 OpenSBI currently supports three different types of firmwares.
 
 Firmware with Dynamic Information (*FW_DYNAMIC*)
@@ -82,9 +87,13 @@ following documents.
 * *[FW_PAYLOAD]*: The *Firmware with Payload (FW_PAYLOAD)* is described in more
   details in the file *fw_payload.md*.
 
+The *OpenSBI firmware header*, which is common to all firmware types, is
+described in *[FW_HEADER]*.
+
 [FW_DYNAMIC]: fw_dynamic.md
 [FW_JUMP]: fw_jump.md
 [FW_PAYLOAD]: fw_payload.md
+[FW_HEADER]: fw_header.md
 
 Providing different payloads to OpenSBI Firmware
 ------------------------------------------------
diff --git a/docs/firmware/fw_header.md b/docs/firmware/fw_header.md
new file mode 100644
index 00000000..7dfca111
--- /dev/null
+++ b/docs/firmware/fw_header.md
@@ -0,0 +1,127 @@
+OpenSBI Firmware Header
+=======================
+
+Every OpenSBI firmware image, regardless of its type (*FW_DYNAMIC*, *FW_JUMP*
+or *FW_PAYLOAD*), starts with a fixed 128-byte header. The header serves two
+purposes:
+
+1. it lets the previous booting stage recognize an OpenSBI firmware image and
+   find out what it was built for, and
+2. it gives the previous booting stage a well-known place inside the image to
+   hand over the boot arguments by patching a few words, instead of setting up
+   the *a0*, *a1* and *a2* registers.
+
+The second point matters when the previous booting stage cannot set up those
+registers at all. A typical example is a system where the previous booting
+stage runs on a dedicated boot processor: it can load and patch the OpenSBI
+image in DRAM and then release the application hart that runs OpenSBI, but it
+can never execute on that hart and therefore cannot place anything in its
+registers.
+
+The first word of the header is a jump over the header, so the entry point of
+the firmware is unchanged: the previous booting stage still jumps to the first
+byte of the image.
+
+Header layout
+-------------
+
+| Offset | Size | Field                                             |
+|--------|------|---------------------------------------------------|
+| 0x00   | 4    | Jump over the header (a 4-byte `j` instruction)   |
+| 0x04   | 4    | Magic value, 'OSBI' (0x4942534f)                  |
+| 0x08   | 4    | Header version, currently 1                       |
+| 0x0c   | 4    | XLEN the firmware was built for, 32 or 64         |
+| 0x10   | 4    | Firmware size in bytes                            |
+| 0x14   | 4    | Flags, see below                                  |
+| 0x18   | 8    | Override value for *a0*                           |
+| 0x20   | 8    | Reserved (upper half of the *a0* value on RV128)  |
+| 0x28   | 8    | Override value for *a1*                           |
+| 0x30   | 8    | Reserved (upper half of the *a1* value on RV128)  |
+| 0x38   | 8    | Override value for *a2*                           |
+| 0x40   | 8    | Reserved (upper half of the *a2* value on RV128)  |
+| 0x48   | 56   | Reserved                                          |
+
+All fields are little-endian. The layout is deliberately the same for RV32 and
+RV64, so the previous booting stage can parse the header, and in particular
+check the *XLEN* field, without knowing the XLEN of the firmware upfront. The
+override values are 8 bytes wide on both; on RV32 only the lower 4 bytes are
+used and the upper 4 bytes must be zero. Each override value is followed by a
+reserved 8-byte slot so that the same layout can be extended to RV128 later.
+
+The *firmware size* field is `_fw_end - _fw_start`, i.e. how much memory the
+firmware image occupies once loaded, including the *.bss* section. Note that
+this is not the size of the flat binary file, which is smaller because
+*.bss* is not stored in the file. OpenSBI also needs some scratch space
+beyond the firmware image, so this field alone does not describe everything
+the previous booting stage has to reserve.
+
+Overriding a0, a1 and a2
+------------------------
+
+The following flags are defined:
+
+| Flag                        | Value | Description                     |
+|-----------------------------|-------|---------------------------------|
+| FW_HEADER_FLAGS_OVERRIDE_A0 | 1 << 0| Override *a0* from offset 0x18  |
+| FW_HEADER_FLAGS_OVERRIDE_A1 | 1 << 1| Override *a1* from offset 0x28  |
+| FW_HEADER_FLAGS_OVERRIDE_A2 | 1 << 2| Override *a2* from offset 0x38  |
+
+For every flag that is set, OpenSBI replaces the corresponding register with
+the value from the header before it looks at the boot arguments. Registers
+whose flag is clear are used exactly as passed by the previous booting stage.
+
+The flags word is zero in a freshly built image, so a previous booting stage
+that already passes *a0*, *a1* and *a2* in registers is unaffected and needs
+to know nothing about the header.
+
+The meaning of the registers is unchanged, so the previous booting stage
+should patch:
+
+* *a0* with the hart id of the hart that will enter OpenSBI,
+* *a1* with the device tree blob address, which must be 8-byte aligned, and
+* *a2* with the address of a *struct fw_dynamic_info*, for a *FW_DYNAMIC*
+  firmware. The structure itself is not part of the OpenSBI image; the
+  previous booting stage has to place it in memory that OpenSBI does not
+  overwrite, which notably excludes the OpenSBI *.bss* section and the
+  scratch space above the firmware.
+
+Patching the header
+-------------------
+
+The header is at the very beginning of the image, so all offsets above are
+relative to the address the image was loaded at. For example, on RV64:
+
+```c
+#define FW_HEADER_MAGIC_VALUE			0x4942534f
+#define FW_HEADER_FLAGS_OVERRIDE_A0		(1 << 0)
+#define FW_HEADER_FLAGS_OVERRIDE_A1		(1 << 1)
+#define FW_HEADER_FLAGS_OVERRIDE_A2		(1 << 2)
+
+struct fw_header {
+	uint32_t	jump;
+	uint32_t	magic;
+	uint32_t	version;
+	uint32_t	xlen;
+	uint32_t	size;
+	uint32_t	flags;
+	uint64_t	override_a0;
+	uint64_t	reserved0;
+	uint64_t	override_a1;
+	uint64_t	reserved1;
+	uint64_t	override_a2;
+	uint64_t	reserved2;
+	uint64_t	reserved[7];
+};
+
+struct fw_header *hdr = (struct fw_header *)opensbi_load_addr;
+
+if (hdr->magic != FW_HEADER_MAGIC_VALUE || hdr->xlen != 64)
+	return -EINVAL;
+
+hdr->override_a0 = hartid;
+hdr->override_a1 = dtb_addr;
+hdr->override_a2 = (uint64_t)dynamic_info;
+hdr->flags = FW_HEADER_FLAGS_OVERRIDE_A0 |
+	     FW_HEADER_FLAGS_OVERRIDE_A1 |
+	     FW_HEADER_FLAGS_OVERRIDE_A2;
+```
-- 
2.43.7


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi