[PATCH v4 23/27] docs/system/riscv: Document Icicle Kit HSS boot

[Bin Meng <bin.meng@processmission.com>]

With all previous changes, the Icicle Kit model is now able to boot
HSS v2026.04.1 through a Buildroot 2026.05 SD image to U-Boot and Linux.

Document the tested source versions and exact commands to build HSS
and Buildroot. Include the sparse resize and GPT relocation steps
needed to make the generated SD image usable by QEMU.

Explain the pinned firmware baseline and include the DDR pattern-test
patch used by the versioned test image. Document the tool check needed
for image-only builds and distinguish the DDR training failure from
the two investigated multi-hart startup races. Describe their visible
symptoms, affected source versions, and all three DTB supply paths.

Also describe the RAM and serial setup, boot stages, and remaining
model limitations.

Signed-off-by: Bin Meng <bin.meng@processmission.com>

---

Changes in v4:
- Use HSS v2026.04.1 and the v1.1.1 test-image recipe
- Select the SoftConsole toolchain explicitly
- Use the generated complete boot-mode-1 eNVM binary

Changes in v3:
- Include the DDR pattern-test patch used by the functional-test image
- Document the image-only FPGA programming tool check
- Describe startup-race symptoms and link the firmware investigations

 docs/system/riscv/microchip-icicle-kit.rst | 129 ++++++++++++++++++++-
 1 file changed, 123 insertions(+), 6 deletions(-)

diff --git a/docs/system/riscv/microchip-icicle-kit.rst b/docs/system/riscv/microchip-icicle-kit.rst
index 138289a913..2159e8e461 100644
--- a/docs/system/riscv/microchip-icicle-kit.rst
+++ b/docs/system/riscv/microchip-icicle-kit.rst
@@ -19,12 +19,14 @@ The ``microchip-icicle-kit`` machine supports the following devices:
 * 4 U54 cores
 * Core Level Interruptor (CLINT)
 * Platform-Level Interrupt Controller (PLIC)
-* L2 Loosely Integrated Memory (L2-LIM)
+* L2 cache controller, including L2-LIM and L2-Zero windows
 * DDR memory controller
+* System controller and system services mailbox
 * 5 MMUARTs
 * 1 DMA controller
 * 2 GEM Ethernet controllers
 * 1 SDHC storage controller
+* 1 Real-Time Clock
 
 The machine has a fixed 2 GiB of RAM. Other memory sizes are rejected.
 The machine defaults to all five harts. ``-smp`` may select two to five harts,
@@ -103,10 +105,125 @@ CAVEATS:
 Running HSS
 -----------
 
-The machine ``microchip-icicle-kit`` used to run the Hart Software Services
-(HSS_), however, the HSS development progressed and the QEMU machine
-implementation lacks behind.  Currently, running the HSS no longer works.
-There is missing support in the clock and memory controller devices.  In
-particular, reading from the SD card does not work.
+The ``microchip-icicle-kit`` machine can boot the Hart Software Services
+(HSS_), which then loads an HSS payload containing U-Boot from an SD card.
+The following flow was tested with HSS v2026.04.1 and Buildroot 2026.05.
+These versions reproduce the firmware and SD card images used by the
+functional test.
+
+Configure HSS for the ``mpfs-icicle-kit-es`` board using its default
+configuration. QEMU provides the software-visible registers and deterministic
+status consumed by the HSS v2026.04.1 DDR initialization and training flow;
+it does not model the electrical properties of DDR training. HSS requires
+the RISC-V bare-metal toolchain supplied by Microchip SoftConsole to be
+available in ``PATH``.
+
+The versioned test image applies the `DDR pattern-test fix`_ before building
+HSS. Without this fix, the DDR state machine can reuse uninitialized
+pattern-test parameters across calls, particularly with the SoftConsole
+GCC 8.3.0 toolchain. Apply the same patch when reproducing the test image:
+
+.. code-block:: bash
+
+  $ git clone https://github.com/polarfire-soc/hart-software-services.git
+  $ cd hart-software-services
+  $ git checkout v2026.04.1
+  $ curl -L -o ddr-pattern-test.patch \
+      https://raw.githubusercontent.com/processmission/qemu-machine-images/v1.1.1/machine/riscv64/microchip-icicle-kit/0001-mpfs_hal-preserve-DDR-pattern-test-parameters.patch
+  $ git apply ddr-pattern-test.patch
+  $ cp boards/mpfs-icicle-kit-es/def_config .config
+  $ echo CONFIG_CC_USE_SOFTCONSOLE=y >> .config
+  $ FPGENPROG=/bin/true make -j$(nproc) BOARD=mpfs-icicle-kit-es
+
+``FPGENPROG=/bin/true`` satisfies the boot-mode programmer's FPGA programming
+tool check when generating an image for QEMU; no physical device is programmed.
+
+The HSS build creates the complete eNVM image, including the 256-byte
+boot header with the image size and per-hart reset vectors. Copy it to
+``build/hss.bin`` for the QEMU command below:
+
+.. code-block:: bash
+
+  $ cp build/bootmode1/hss-envm-wrapper-bm1-p0.bin build/hss.bin
+
+Do not pass ``build/hss-envm-wrapper.bin`` directly to QEMU. That file starts
+at eNVM offset 0x100 and does not contain the boot header with the image size
+and per-hart reset vectors.
+
+Build the SD card image with the tested Buildroot version:
+
+.. code-block:: bash
+
+  $ git clone https://gitlab.com/buildroot.org/buildroot.git
+  $ cd buildroot
+  $ git checkout 2026.05
+  $ make microchip_mpfs_icicle_defconfig
+  $ make
+
+This produces ``output/images/sdcard.img`` with three GPT partitions:
+
+* An HSS ``payload.bin`` containing U-Boot.
+* A FAT partition containing ``boot.scr`` and the kernel FIT image.
+* An ext4 Linux root filesystem.
+
+The QEMU SD card model requires a power-of-two image size. Make a sparse
+4 GiB working copy and relocate its backup GPT to the new end of the image:
+
+.. code-block:: bash
+
+  $ cp --reflink=auto output/images/sdcard.img sdcard.img
+  $ truncate -s 4G sdcard.img
+  $ sgdisk -e sdcard.img
+  $ sgdisk -v sdcard.img
+
+The Icicle Kit firmware device tree in the FIT image describes 2 GiB of RAM,
+matching the machine's fixed RAM size. The command below keeps ``-m 2G``
+explicit.  Attach the image as an SD card and route both board serial ports:
+
+.. code-block:: bash
+
+  $ qemu-system-riscv64 \
+      -M microchip-icicle-kit -smp 5 -m 2G \
+      -bios path/to/hss/build/hss.bin \
+      -drive if=sd,file=path/to/sdcard.img,format=raw \
+      -display none \
+      -serial file:hss.log \
+      -serial stdio \
+      -no-reboot
+
+HSS writes to MMUART0, which the command records in ``hss.log``. U-Boot and
+Linux use MMUART1, which remains connected to the terminal. A successful
+boot proceeds through HSS payload loading, U-Boot, the Linux kernel, and the
+login prompt from the root filesystem on the third partition.
+
+Known limitations
+-----------------
+
+* Two potential multi-hart startup races were reproduced with HSS v2024.06
+  under QEMU. The affected code paths are also present in the inspected
+  HSS v2026.04.1 sources; they have not been reproduced on physical hardware.
+  See `HSS startup races`_ for the investigation and affected source versions.
+
+  * In the eNVM wrapper, a late U54 can clear the decompression completion
+    flag after E51 has set it. A boot can stop before all harts enter HSS,
+    with the affected U54s waiting in ``wait_for_decompress_done``.
+  * After DDR training and payload loading, a U54 other than the configured
+    payload owner can win the OpenSBI coldboot lottery. With the minimal DTB
+    describing only ``cpu@1``, U-Boot on another hart can repeatedly print
+    ``Could not initialize timer (err -19)`` and ``resetting ...``.
+
+* Without the DDR pattern-test fix above, some GCC 8.3.0 builds remain at
+  60% DDR training progress while QEMU keeps running. This is a separate
+  firmware parameter-lifetime issue; see `DDR pattern-test issue`_.
+* The SD card model requires the raw image size to be a power of two. Keep
+  the backup GPT header at the end when resizing the image.
+* The machine does not generate an Icicle Kit device tree. Firmware boot can
+  obtain a DTB built into HSS, included in an HSS payload, or supplied by
+  subsequent software, such as U-Boot or a kernel FIT image. The DTB must
+  describe the selected hart topology and the modeled reference-design DDR
+  windows. Direct kernel boot must use ``-dtb`` as described above.
 
 .. _HSS: https://github.com/polarfire-soc/hart-software-services
+.. _DDR pattern-test fix: https://github.com/processmission/qemu-machine-images/blob/v1.1.1/machine/riscv64/microchip-icicle-kit/0001-mpfs_hal-preserve-DDR-pattern-test-parameters.patch
+.. _DDR pattern-test issue: https://github.com/polarfire-soc/platform/issues/36
+.. _HSS startup races: https://github.com/polarfire-soc/hart-software-services/issues/106
-- 
2.53.0