[PATCH v4 0/5] Fully support standalone Clang/LLVM toolchains

[PATCH v4 0/5] Fully support standalone Clang/LLVM toolchains

[Jessica Clarke]

This patch series is comprised of five parts.

The first patch fixes a warning seen when building with LLVM due to a
bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a
position-independent binary using LLD that may or may not be an LLD bug
(the exact meaning of -N/--omagic isn't clear) but can easily be worked
around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not
honouring -pie for bare-metal binaries as it's normally not something
you want (arguably that should be an error though, or passed on, since
just giving a -Wunused-command-line-argument warning can get lost, and
may be disabled).

The fourth patch adds build system support for using Clang and LLVM
binutils, provided your Clang is able to locate a libgcc.a in its search
path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt
(libclang_rt.builtins.<arch>.a). We could change the Makefile to not
hard-code -lgcc and instead use -print-libgcc-file-name, but that still
requires a bare metal compiler-rt built for the right -march/-mabi to be
present, which is often not the case. Moreover, we need very little from
libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit
division. Thus, the fifth patch vendors part of FreeBSD's libquad and
stops linking against libgcc entirely, allowing OpenSBI to be built with
just a cross-compiler. This means that building with any distro-provided
LLVM just works, as does compiling with the system Clang compiler and
LLD linker on FreeBSD without any external packages needed (beyond GNU
make).

Changes in v4:
  * Avoid read-only dynamic relocation error with LLD
  * Support FW_PIC with LLD when using a bare-metal triple

Changes in v3:
  * Fixed the binding warning seen when building with LLVM
  * Provide a default -target when CROSS_COMPILE isn't set so the
    variable isn't normally required to be set when cross-compiling with
    LLVM (but is still honoured if set).
  * Pass -fuse=bfd when not using LLD to ensure an ld.bfd gets correctly
    picked up over a plain ld when ld is LLD. Now we always explicitly
    request Clang and GCC use the linker type corresponding to LD. This
    allows BFD to be used on FreeBSD for testing purposes despite LLD
    being the system linker.

Changes in v2:
  * Add documentation to README.md
  * Pass -fuse-ld=lld to Clang when using LLD
  * Modify commit message of first commit to explain -N -> -Wl,-N change
  * Bring back the old ?= uses for the non-CROSS_COMPILE (and, now,
    non-LLVM) case; whilst I still think that's how it should be, it's
    not required for this patch, just related cleanup I did to reduce
    the complexity of adding the LLVM case.

Jessica Clarke (5):
  fw_base: Don't mark fw_platform_init as both global and weak
  fw_base: Put data in .data rather than .text
  firmware: Explicitly pass -pie to the linker, not just the driver
  Makefile: Support building with Clang and LLVM binutils
  Drop dependency on libgcc by importing part of FreeBSD's libquad

 Makefile                              |  65 +++++-
 README.md                             |  43 +++-
 firmware/fw_base.S                    |   2 +-
 firmware/objects.mk                   |   2 +-
 lib/utils/libquad/divdi3.c            |  64 ++++++
 lib/utils/libquad/include/limits.h    |  12 ++
 lib/utils/libquad/include/sys/cdefs.h |  12 ++
 lib/utils/libquad/include/sys/types.h |  25 +++
 lib/utils/libquad/moddi3.c            |  66 ++++++
 lib/utils/libquad/objects.mk          |  14 ++
 lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
 lib/utils/libquad/quad.h              | 105 ++++++++++
 lib/utils/libquad/udivdi3.c           |  52 +++++
 lib/utils/libquad/umoddi3.c           |  54 +++++
 14 files changed, 784 insertions(+), 10 deletions(-)
 create mode 100644 lib/utils/libquad/divdi3.c
 create mode 100644 lib/utils/libquad/include/limits.h
 create mode 100644 lib/utils/libquad/include/sys/cdefs.h
 create mode 100644 lib/utils/libquad/include/sys/types.h
 create mode 100644 lib/utils/libquad/moddi3.c
 create mode 100644 lib/utils/libquad/objects.mk
 create mode 100644 lib/utils/libquad/qdivrem.c
 create mode 100644 lib/utils/libquad/quad.h
 create mode 100644 lib/utils/libquad/udivdi3.c
 create mode 100644 lib/utils/libquad/umoddi3.c

-- 
2.31.0

[PATCH v4 1/5] fw_base: Don't mark fw_platform_init as both global and weak

[Jessica Clarke]

These are mutually exclusive. GNU as and LLVM both let later binding
directives override earlier ones so this works as intended, but LLVM 12
turned this into a warning as there's no good reason to do such a thing
and could be a potential bug. Thus, remove the redundant and incorrect
.globl directive for fw_platform_init.
---
 firmware/fw_base.S | 1 -
 1 file changed, 1 deletion(-)

diff --git a/firmware/fw_base.S b/firmware/fw_base.S
index a5ce946..ee2a51b 100644
--- a/firmware/fw_base.S
+++ b/firmware/fw_base.S
@@ -553,7 +553,6 @@ _start_hang:
 
 	.section .entry, "ax", %progbits
 	.align 3
-	.globl fw_platform_init
 	.weak fw_platform_init
 fw_platform_init:
 	add	a0, a1, zero
-- 
2.31.0

[PATCH v4 2/5] fw_base: Put data in .data rather than .text

[Jessica Clarke]

The -N linker option is supposed to make .text writable, but GNU ld and
LLD differ in interpreting what that means. GNU ld will happily let you
have relocations in it, but LLD will see that the input section is
read-only (even though the output section is writable) and give an
error. It's unclear if either of them intend to have that behaviour in
this edge case, but regardless there's no reason not to just put the
data in a writable .data section.
---
 firmware/fw_base.S | 1 +
 1 file changed, 1 insertion(+)

diff --git a/firmware/fw_base.S b/firmware/fw_base.S
index ee2a51b..f8aea05 100644
--- a/firmware/fw_base.S
+++ b/firmware/fw_base.S
@@ -501,6 +501,7 @@ _skip_trap_exit_rv32_hyp:
 	/* We don't expect to reach here hence just hang */
 	j	_start_hang
 
+	.data
 	.align 3
 #ifdef FW_PIC
 _runtime_offset:
-- 
2.31.0

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Jessica Clarke]

When using Clang with a bare-metal triple, -pie does not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as desired, so do so
to fully support FW_PIC with Clang, including when linking with LLD.
---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/firmware/objects.mk b/firmware/objects.mk
index ce91c2f..3bc83cd 100644
--- a/firmware/objects.mk
+++ b/firmware/objects.mk
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y +=	-DFW_PIC
 firmware-asflags-y  +=	-fpic
 firmware-cflags-y   +=	-fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif
 
 ifdef FW_TEXT_START
-- 
2.31.0

[PATCH v4 4/5] Makefile: Support building with Clang and LLVM binutils

[Jessica Clarke]

This is intended to mirror the Linux kernel. Building with CC=clang will
use Clang as the compiler but default to using the existing binutils.
Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the
linker, Clang will not, and so in order to support both compilers we
must use -Wl, to forward it to the linker as is required for most other
linker options.

Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
---
 Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
 README.md | 43 ++++++++++++++++++++++++++++++++++--
 2 files changed, 100 insertions(+), 8 deletions(-)

diff --git a/Makefile b/Makefile
index 6b64205..50b9f89 100644
--- a/Makefile
+++ b/Makefile
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC		=	clang
+AR		=	llvm-ar
+LD		=	ld.lld
+OBJCOPY		=	llvm-objcopy
+else
 ifdef CROSS_COMPILE
 CC		=	$(CROSS_COMPILE)gcc
-CPP		=	$(CROSS_COMPILE)cpp
 AR		=	$(CROSS_COMPILE)ar
 LD		=	$(CROSS_COMPILE)ld
 OBJCOPY		=	$(CROSS_COMPILE)objcopy
 else
 CC		?=	gcc
-CPP		?=	cpp
 AR		?=	ar
 LD		?=	ld
 OBJCOPY		?=	objcopy
 endif
+endif
+CPP		=	$(CC) -E
 AS		=	$(CC)
 DTC		=	dtc
 
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG	=	y
+else
+CC_IS_CLANG	=	n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD	=	y
+else
+LD_IS_LLD	=	n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET	=	-target $(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)
 OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
 OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
+endif
 
 # Setup platform XLEN
 ifndef PLATFORM_RISCV_XLEN
@@ -106,6 +134,12 @@ ifndef PLATFORM_RISCV_XLEN
   endif
 endif
 
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET	=	-target riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+
 # Setup list of objects.mk files
 ifdef PLATFORM
 platform-object-mks=$(shell if [ -d $(platform_src_dir)/ ]; then find $(platform_src_dir) -iname "objects.mk" | sort -r; fi)
@@ -194,7 +228,11 @@ else
 endif
 
 # Setup compilation commands flags
-GENFLAGS	=	-I$(platform_src_dir)/include
+ifeq ($(CC_IS_CLANG),y)
+GENFLAGS	+=	$(CLANG_TARGET)
+GENFLAGS	+=	-Wno-unused-command-line-argument
+endif
+GENFLAGS	+=	-I$(platform_src_dir)/include
 GENFLAGS	+=	-I$(include_dir)
 ifneq ($(OPENSBI_VERSION_GIT),)
 GENFLAGS	+=	-DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\""
@@ -208,6 +246,9 @@ CFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls
 CFLAGS		+=	-mno-save-restore -mstrict-align
 CFLAGS		+=	-mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS		+=	-mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+ifeq ($(LD_IS_LLD),y)
+CFLAGS		+=	-mno-relax
+endif
 CFLAGS		+=	$(GENFLAGS)
 CFLAGS		+=	$(platform-cflags-y)
 CFLAGS		+=	-fno-pie -no-pie
@@ -222,18 +263,30 @@ ASFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls
 ASFLAGS		+=	-mno-save-restore -mstrict-align
 ASFLAGS		+=	-mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 ASFLAGS		+=	-mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+ifeq ($(LD_IS_LLD),y)
+ASFLAGS		+=	-mno-relax
+endif
 ASFLAGS		+=	$(GENFLAGS)
 ASFLAGS		+=	$(platform-asflags-y)
 ASFLAGS		+=	$(firmware-asflags-y)
 
 ARFLAGS		=	rcs
 
-ELFFLAGS	+=	-Wl,--build-id=none -N -static-libgcc -lgcc
+ifeq ($(LD_IS_LLD),y)
+ELFFLAGS	+=	-fuse-ld=lld
+else
+ELFFLAGS	+=	-fuse-ld=bfd
+endif
+ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
 ELFFLAGS	+=	$(platform-ldflags-y)
 ELFFLAGS	+=	$(firmware-ldflags-y)
 
 MERGEFLAGS	+=	-r
+ifeq ($(LD_IS_LLD),y)
+MERGEFLAGS	+=	-b elf
+else
 MERGEFLAGS	+=	-b elf$(PLATFORM_RISCV_XLEN)-littleriscv
+endif
 MERGEFLAGS	+=	-m elf$(PLATFORM_RISCV_XLEN)lriscv
 
 DTSCPPFLAGS	=	$(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp
diff --git a/README.md b/README.md
index 03c02fb..d5d6ab7 100644
--- a/README.md
+++ b/README.md
@@ -96,8 +96,13 @@ Required Toolchain
 ------------------
 
 OpenSBI can be compiled natively or cross-compiled on a x86 host. For
-cross-compilation, you can build your own toolchain or just download
-a prebuilt one from the [Bootlin toolchain repository].
+cross-compilation, you can build your own toolchain, download a prebuilt one
+from the [Bootlin toolchain repository] or install a distribution-provided
+toolchain; if you opt to use LLVM/Clang, most distribution toolchains will
+support cross-compiling for RISC-V using the same toolchain as your native
+LLVM/Clang toolchain due to LLVM's ability to support multiple backends in the
+same binary, so is often an easy way to obtain a working cross-compilation
+toolchain.
 
 Please note that only a 64-bit version of the toolchain is available in
 the Bootlin toolchain repository for now.
@@ -202,6 +207,40 @@ export PLATFORM_RISCV_XLEN=32
 
 will generate 32-bit OpenSBI images. And vice vesa.
 
+Building with Clang/LLVM
+------------------------
+
+OpenSBI can also be built with Clang/LLVM. To build with just Clang but keep
+the default binutils (which will still use the *CROSS_COMPILE* prefix if
+defined), override the *CC* make variable with:
+```
+make CC=clang
+```
+
+To build with a full LLVM-based toolchain, not just Clang, enable the *LLVM*
+option with:
+```
+make LLVM=1
+```
+
+When using Clang, *CROSS_COMPILE* often does not need to be defined unless
+using GNU binutils with prefixed binary names. *PLATFORM_RISCV_XLEN* will be
+used to infer a default triple to pass to Clang, so if *PLATFORM_RISCV_XLEN*
+itself defaults to an undesired value then prefer setting that rather than the
+full triple via *CROSS_COMPILE*. If *CROSS_COMPILE* is nonetheless defined,
+rather than being used as a prefix for the executable name, it will instead be
+passed via the `-target` option with the trailing `-` removed, so must be a
+valid triple.
+
+These can also be mixed; for example using a GCC cross-compiler but LLVM
+binutils would be:
+```
+make CC=riscv64-unknown-elf-gcc LLVM=1
+```
+
+These variables must be passed for all the make invocations described in this
+document.
+
 Contributing to OpenSBI
 -----------------------
 
-- 
2.31.0

[PATCH v4 5/5] Drop dependency on libgcc by importing part of FreeBSD's libquad

[Jessica Clarke]

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains
bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins
given every compiler is a cross-compiler for every target and so you
would need a silly number of builds of it, with only the native library
available; only vendor-provided Clang toolchains specifically for bare
metal cross-compiling are likely to provide it.

Thus, import part of FreeBSD's implementation of the division support
functions needed and stop linking against libgcc.

Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
---
 Makefile                              |   2 +-
 lib/utils/libquad/divdi3.c            |  64 ++++++
 lib/utils/libquad/include/limits.h    |  12 ++
 lib/utils/libquad/include/sys/cdefs.h |  12 ++
 lib/utils/libquad/include/sys/types.h |  25 +++
 lib/utils/libquad/moddi3.c            |  66 ++++++
 lib/utils/libquad/objects.mk          |  14 ++
 lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
 lib/utils/libquad/quad.h              | 105 ++++++++++
 lib/utils/libquad/udivdi3.c           |  52 +++++
 lib/utils/libquad/umoddi3.c           |  54 +++++
 11 files changed, 683 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 50b9f89..0672558 100644
--- a/Makefile
+++ b/Makefile
@@ -277,7 +277,7 @@ ELFFLAGS	+=	-fuse-ld=lld
 else
 ELFFLAGS	+=	-fuse-ld=bfd
 endif
-ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
+ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N
 ELFFLAGS	+=	$(platform-ldflags-y)
 ELFFLAGS	+=	$(firmware-ldflags-y)
 
diff --git a/lib/utils/libquad/divdi3.c b/lib/utils/libquad/divdi3.c
new file mode 100644
index 0000000..c5b4a37
--- /dev/null
+++ b/lib/utils/libquad/divdi3.c
@@ -0,0 +1,64 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *	The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)divdi3.c	8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Divide two signed quads.
+ * ??? if -1/2 should produce -1 on this machine, this code is wrong
+ */
+quad_t
+__divdi3(quad_t a, quad_t b)
+{
+	u_quad_t ua, ub, uq;
+	int neg;
+
+	if (a < 0)
+		ua = -(u_quad_t)a, neg = 1;
+	else
+		ua = a, neg = 0;
+	if (b < 0)
+		ub = -(u_quad_t)b, neg ^= 1;
+	else
+		ub = b;
+	uq = __qdivrem(ua, ub, (u_quad_t *)0);
+	return (neg ? -uq : uq);
+}
diff --git a/lib/utils/libquad/include/limits.h b/lib/utils/libquad/include/limits.h
new file mode 100644
index 0000000..9c69d27
--- /dev/null
+++ b/lib/utils/libquad/include/limits.h
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <jrtc27@jrtc27.com>
+ */
+
+#ifndef __LIMITS_H__
+#define __LIMITS_H__
+
+#define CHAR_BIT 8
+
+#endif
diff --git a/lib/utils/libquad/include/sys/cdefs.h b/lib/utils/libquad/include/sys/cdefs.h
new file mode 100644
index 0000000..e8d6faa
--- /dev/null
+++ b/lib/utils/libquad/include/sys/cdefs.h
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <jrtc27@jrtc27.com>
+ */
+
+#ifndef __SYS_CDEFS_H__
+#define __SYS_CDEFS_H__
+
+#define __FBSDID(s) struct __hack
+
+#endif
diff --git a/lib/utils/libquad/include/sys/types.h b/lib/utils/libquad/include/sys/types.h
new file mode 100644
index 0000000..e372437
--- /dev/null
+++ b/lib/utils/libquad/include/sys/types.h
@@ -0,0 +1,25 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <jrtc27@jrtc27.com>
+ */
+
+#ifndef __SYS_TYPES_H__
+#define __SYS_TYPES_H__
+
+#include <sbi/sbi_types.h>
+
+typedef unsigned long u_long;
+
+typedef int64_t quad_t;
+typedef uint64_t u_quad_t;
+
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+#define _QUAD_LOWWORD 1
+#define _QUAD_HIGHWORD 0
+#else
+#define _QUAD_LOWWORD 0
+#define _QUAD_HIGHWORD 1
+#endif
+
+#endif
diff --git a/lib/utils/libquad/moddi3.c b/lib/utils/libquad/moddi3.c
new file mode 100644
index 0000000..fc13b4d
--- /dev/null
+++ b/lib/utils/libquad/moddi3.c
@@ -0,0 +1,66 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *	The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)moddi3.c	8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Return remainder after dividing two signed quads.
+ *
+ * XXX
+ * If -1/2 should produce -1 on this machine, this code is wrong.
+ */
+quad_t
+__moddi3(quad_t a, quad_t b)
+{
+	u_quad_t ua, ub, ur;
+	int neg;
+
+	if (a < 0)
+		ua = -(u_quad_t)a, neg = 1;
+	else
+		ua = a, neg = 0;
+	if (b < 0)
+		ub = -(u_quad_t)b;
+	else
+		ub = b;
+	(void)__qdivrem(ua, ub, &ur);
+	return (neg ? -ur : ur);
+}
diff --git a/lib/utils/libquad/objects.mk b/lib/utils/libquad/objects.mk
new file mode 100644
index 0000000..5e53cd0
--- /dev/null
+++ b/lib/utils/libquad/objects.mk
@@ -0,0 +1,14 @@
+#
+# SPDX-License-Identifier: BSD-2-Clause
+#
+# Copyright (c) 2021 Jessica Clarke <jrtc27@jrtc27.com>
+#
+
+ifeq ($(PLATFORM_RISCV_XLEN),32)
+libsbiutils-objs-y += libquad/divdi3.o
+libsbiutils-objs-y += libquad/moddi3.o
+libsbiutils-objs-y += libquad/qdivrem.o
+libsbiutils-objs-y += libquad/udivdi3.o
+libsbiutils-objs-y += libquad/umoddi3.o
+libsbiutils-genflags-y += -I$(libsbiutils_dir)/libquad/include
+endif
diff --git a/lib/utils/libquad/qdivrem.c b/lib/utils/libquad/qdivrem.c
new file mode 100644
index 0000000..ea09e7f
--- /dev/null
+++ b/lib/utils/libquad/qdivrem.c
@@ -0,0 +1,278 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *	The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)qdivrem.c	8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+/*
+ * Multiprecision divide.  This algorithm is from Knuth vol. 2 (2nd ed),
+ * section 4.3.1, pp. 257--259.
+ */
+
+#include "quad.h"
+
+#define	B	(1L << HALF_BITS)	/* digit base */
+
+/* Combine two `digits' to make a single two-digit number. */
+#define	COMBINE(a, b) (((u_long)@ << HALF_BITS) | (b))
+
+/* select a type for digits in base B: use unsigned short if they fit */
+#if ULONG_MAX == 0xffffffff && USHRT_MAX >= 0xffff
+typedef unsigned short digit;
+#else
+typedef u_long digit;
+#endif
+
+/*
+ * Shift p[0]..p[len] left `sh' bits, ignoring any bits that
+ * `fall out' the left (there never will be any such anyway).
+ * We may assume len >= 0.  NOTE THAT THIS WRITES len+1 DIGITS.
+ */
+static void
+shl(digit *p, int len, int sh)
+{
+	int i;
+
+	for (i = 0; i < len; i++)
+		p[i] = LHALF(p[i] << sh) | (p[i + 1] >> (HALF_BITS - sh));
+	p[i] = LHALF(p[i] << sh);
+}
+
+/*
+ * __qdivrem(u, v, rem) returns u/v and, optionally, sets *rem to u%v.
+ *
+ * We do this in base 2-sup-HALF_BITS, so that all intermediate products
+ * fit within u_long.  As a consequence, the maximum length dividend and
+ * divisor are 4 `digits' in this base (they are shorter if they have
+ * leading zeros).
+ */
+u_quad_t
+__qdivrem(u_quad_t uq, u_quad_t vq, u_quad_t *arq)
+{
+	union uu tmp;
+	digit *u, *v, *q;
+	digit v1, v2;
+	u_long qhat, rhat, t;
+	int m, n, d, j, i;
+	digit uspace[5], vspace[5], qspace[5];
+
+	/*
+	 * Take care of special cases: divide by zero, and u < v.
+	 */
+	if (vq == 0) {
+		/* divide by zero. */
+		static volatile const unsigned int zero = 0;
+
+		tmp.ul[H] = tmp.ul[L] = 1 / zero;
+		if (arq)
+			*arq = uq;
+		return (tmp.q);
+	}
+	if (uq < vq) {
+		if (arq)
+			*arq = uq;
+		return (0);
+	}
+	u = &uspace[0];
+	v = &vspace[0];
+	q = &qspace[0];
+
+	/*
+	 * Break dividend and divisor into digits in base B, then
+	 * count leading zeros to determine m and n.  When done, we
+	 * will have:
+	 *	u = (u[1]u[2]...u[m+n]) sub B
+	 *	v = (v[1]v[2]...v[n]) sub B
+	 *	v[1] != 0
+	 *	1 < n <= 4 (if n = 1, we use a different division algorithm)
+	 *	m >= 0 (otherwise u < v, which we already checked)
+	 *	m + n = 4
+	 * and thus
+	 *	m = 4 - n <= 2
+	 */
+	tmp.uq = uq;
+	u[0] = 0;
+	u[1] = HHALF(tmp.ul[H]);
+	u[2] = LHALF(tmp.ul[H]);
+	u[3] = HHALF(tmp.ul[L]);
+	u[4] = LHALF(tmp.ul[L]);
+	tmp.uq = vq;
+	v[1] = HHALF(tmp.ul[H]);
+	v[2] = LHALF(tmp.ul[H]);
+	v[3] = HHALF(tmp.ul[L]);
+	v[4] = LHALF(tmp.ul[L]);
+	for (n = 4; v[1] == 0; v++) {
+		if (--n == 1) {
+			u_long rbj;	/* r*B+u[j] (not root boy jim) */
+			digit q1, q2, q3, q4;
+
+			/*
+			 * Change of plan, per exercise 16.
+			 *	r = 0;
+			 *	for j = 1..4:
+			 *		q[j] = floor((r*B + u[j]) / v),
+			 *		r = (r*B + u[j]) % v;
+			 * We unroll this completely here.
+			 */
+			t = v[2];	/* nonzero, by definition */
+			q1 = u[1] / t;
+			rbj = COMBINE(u[1] % t, u[2]);
+			q2 = rbj / t;
+			rbj = COMBINE(rbj % t, u[3]);
+			q3 = rbj / t;
+			rbj = COMBINE(rbj % t, u[4]);
+			q4 = rbj / t;
+			if (arq)
+				*arq = rbj % t;
+			tmp.ul[H] = COMBINE(q1, q2);
+			tmp.ul[L] = COMBINE(q3, q4);
+			return (tmp.q);
+		}
+	}
+
+	/*
+	 * By adjusting q once we determine m, we can guarantee that
+	 * there is a complete four-digit quotient@&qspace[1] when
+	 * we finally stop.
+	 */
+	for (m = 4 - n; u[1] == 0; u++)
+		m--;
+	for (i = 4 - m; --i >= 0;)
+		q[i] = 0;
+	q += 4 - m;
+
+	/*
+	 * Here we run Program D, translated from MIX to C and acquiring
+	 * a few minor changes.
+	 *
+	 * D1: choose multiplier 1 << d to ensure v[1] >= B/2.
+	 */
+	d = 0;
+	for (t = v[1]; t < B / 2; t <<= 1)
+		d++;
+	if (d > 0) {
+		shl(&u[0], m + n, d);		/* u <<= d */
+		shl(&v[1], n - 1, d);		/* v <<= d */
+	}
+	/*
+	 * D2: j = 0.
+	 */
+	j = 0;
+	v1 = v[1];	/* for D3 -- note that v[1..n] are constant */
+	v2 = v[2];	/* for D3 */
+	do {
+		digit uj0, uj1, uj2;
+
+		/*
+		 * D3: Calculate qhat (\^q, in TeX notation).
+		 * Let qhat = min((u[j]*B + u[j+1])/v[1], B-1), and
+		 * let rhat = (u[j]*B + u[j+1]) mod v[1].
+		 * While rhat < B and v[2]*qhat > rhat*B+u[j+2],
+		 * decrement qhat and increase rhat correspondingly.
+		 * Note that if rhat >= B, v[2]*qhat < rhat*B.
+		 */
+		uj0 = u[j + 0];	/* for D3 only -- note that u[j+...] change */
+		uj1 = u[j + 1];	/* for D3 only */
+		uj2 = u[j + 2];	/* for D3 only */
+		if (uj0 == v1) {
+			qhat = B;
+			rhat = uj1;
+			goto qhat_too_big;
+		} else {
+			u_long n = COMBINE(uj0, uj1);
+			qhat = n / v1;
+			rhat = n % v1;
+		}
+		while (v2 * qhat > COMBINE(rhat, uj2)) {
+	qhat_too_big:
+			qhat--;
+			if ((rhat += v1) >= B)
+				break;
+		}
+		/*
+		 * D4: Multiply and subtract.
+		 * The variable `t' holds any borrows across the loop.
+		 * We split this up so that we do not require v[0] = 0,
+		 * and to eliminate a final special case.
+		 */
+		for (t = 0, i = n; i > 0; i--) {
+			t = u[i + j] - v[i] * qhat - t;
+			u[i + j] = LHALF(t);
+			t = (B - HHALF(t)) & (B - 1);
+		}
+		t = u[j] - t;
+		u[j] = LHALF(t);
+		/*
+		 * D5: test remainder.
+		 * There is a borrow if and only if HHALF(t) is nonzero;
+		 * in that (rare) case, qhat was too large (by exactly 1).
+		 * Fix it by adding v[1..n] to u[j..j+n].
+		 */
+		if (HHALF(t)) {
+			qhat--;
+			for (t = 0, i = n; i > 0; i--) { /* D6: add back. */
+				t += u[i + j] + v[i];
+				u[i + j] = LHALF(t);
+				t = HHALF(t);
+			}
+			u[j] = LHALF(u[j] + t);
+		}
+		q[j] = qhat;
+	} while (++j <= m);		/* D7: loop on j. */
+
+	/*
+	 * If caller wants the remainder, we have to calculate it as
+	 * u[m..m+n] >> d (this is at most n digits and thus fits in
+	 * u[m+1..m+n], but we may need more source digits).
+	 */
+	if (arq) {
+		if (d) {
+			for (i = m + n; i > m; --i)
+				u[i] = (u[i] >> d) |
+				    LHALF(u[i - 1] << (HALF_BITS - d));
+			u[i] = 0;
+		}
+		tmp.ul[H] = COMBINE(uspace[1], uspace[2]);
+		tmp.ul[L] = COMBINE(uspace[3], uspace[4]);
+		*arq = tmp.q;
+	}
+
+	tmp.ul[H] = COMBINE(qspace[1], qspace[2]);
+	tmp.ul[L] = COMBINE(qspace[3], qspace[4]);
+	return (tmp.q);
+}
diff --git a/lib/utils/libquad/quad.h b/lib/utils/libquad/quad.h
new file mode 100644
index 0000000..a9f8447
--- /dev/null
+++ b/lib/utils/libquad/quad.h
@@ -0,0 +1,105 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *	The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ *@Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ *	@(#)quad.h	8.1 (Berkeley) 6/4/93
+ * $FreeBSD$
+ */
+
+/*
+ * Quad arithmetic.
+ *
+ * This library makes the following assumptions:
+ *
+ *  - The type long long (aka quad_t) exists.
+ *
+ *  - A quad variable is exactly twice as long as `long'.
+ *
+ *  - The machine's arithmetic is two's complement.
+ *
+ * This library can provide 128-bit arithmetic on a machine with 128-bit
+ * quads and 64-bit longs, for instance, or 96-bit arithmetic on machines
+ * with 48-bit longs.
+ */
+
+#include <sys/types.h>
+#include <limits.h>
+
+/*
+ * Depending on the desired operation, we view a `long long' (aka quad_t) in
+ * one or more of the following formats.
+ */
+union uu {
+	quad_t	q;		/* as a (signed) quad */
+	quad_t	uq;		/* as an unsigned quad */
+	long	sl[2];		/* as two signed longs */
+	u_long	ul[2];		/* as two unsigned longs */
+};
+
+/*
+ * Define high and low longwords.
+ */
+#define	H		_QUAD_HIGHWORD
+#define	L		_QUAD_LOWWORD
+
+/*
+ * Total number of bits in a quad_t and in the pieces that make it up.
+ * These are used for shifting, and also below for halfword extraction
+ * and assembly.
+ */
+#define	QUAD_BITS	(sizeof(quad_t) * CHAR_BIT)
+#define	LONG_BITS	(sizeof(long) * CHAR_BIT)
+#define	HALF_BITS	(sizeof(long) * CHAR_BIT / 2)
+
+/*
+ * Extract high and low shortwords from longword, and move low shortword of
+ * longword to upper half of long, i.e., produce the upper longword of
+ * ((quad_t)(x) << (number_of_bits_in_long/2)).  (`x' must actually be u_long.)
+ *
+ * These are used in the multiply code, to split a longword into upper
+ * and lower halves, and to reassemble a product as a quad_t, shifted left
+ * (sizeof(long)*CHAR_BIT/2).
+ */
+#define	HHALF(x)	((x) >> HALF_BITS)
+#define	LHALF(x)	((x) & ((1L << HALF_BITS) - 1))
+#define	LHUP(x)		((x) << HALF_BITS)
+
+int		__cmpdi2(quad_t a, quad_t b);
+quad_t		__divdi3(quad_t a, quad_t b);
+quad_t		__moddi3(quad_t a, quad_t b);
+u_quad_t	__qdivrem(u_quad_t u, u_quad_t v, u_quad_t *rem);
+int		__ucmpdi2(u_quad_t a, u_quad_t b);
+u_quad_t	__udivdi3(u_quad_t a, u_quad_t b);
+u_quad_t	__umoddi3(u_quad_t a, u_quad_t b);
+
+typedef unsigned int	qshift_t;
diff --git a/lib/utils/libquad/udivdi3.c b/lib/utils/libquad/udivdi3.c
new file mode 100644
index 0000000..0e6f27e
--- /dev/null
+++ b/lib/utils/libquad/udivdi3.c
@@ -0,0 +1,52 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *	The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ *@Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)udivdi3.c	8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Divide two unsigned quads.
+ */
+u_quad_t
+__udivdi3(u_quad_t a, u_quad_t b)
+{
+
+	return (__qdivrem(a, b, (u_quad_t *)0));
+}
diff --git a/lib/utils/libquad/umoddi3.c b/lib/utils/libquad/umoddi3.c
new file mode 100644
index 0000000..7f45134
--- /dev/null
+++ b/lib/utils/libquad/umoddi3.c
@@ -0,0 +1,54 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *	The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)umoddi3.c	8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Return remainder after dividing two unsigned quads.
+ */
+u_quad_t
+__umoddi3(u_quad_t a, u_quad_t b)
+{
+	u_quad_t r;
+
+	(void)__qdivrem(a, b, &r);
+	return (r);
+}
-- 
2.31.0

[PATCH v4 0/5] Fully support standalone Clang/LLVM toolchains

[Jessica Clarke]

On 9 Jul 2021, at 20:34, Jessica Clarke <jrtc27@jrtc27.com> wrote:
> 
> This patch series is comprised of five parts.
> 
> The first patch fixes a warning seen when building with LLVM due to a
> bogus combination of assembly directives.
> 
> The second patch fixes errors seen when trying to build OpenSBI as a
> position-independent binary using LLD that may or may not be an LLD bug
> (the exact meaning of -N/--omagic isn't clear) but can easily be worked
> around without any issue either way.
> 
> The third patch bypasses the Clang driver's helpful nature of not
> honouring -pie for bare-metal binaries as it's normally not something
> you want (arguably that should be an error though, or passed on, since
> just giving a -Wunused-command-line-argument warning can get lost, and
> may be disabled).
> 
> The fourth patch adds build system support for using Clang and LLVM
> binutils, provided your Clang is able to locate a libgcc.a in its search
> path.
> 
> However, pure LLVM toolchains do not use libgcc, they use compiler-rt
> (libclang_rt.builtins.<arch>.a). We could change the Makefile to not
> hard-code -lgcc and instead use -print-libgcc-file-name, but that still
> requires a bare metal compiler-rt built for the right -march/-mabi to be
> present, which is often not the case. Moreover, we need very little from
> libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit
> division. Thus, the fifth patch vendors part of FreeBSD's libquad and
> stops linking against libgcc entirely, allowing OpenSBI to be built with
> just a cross-compiler. This means that building with any distro-provided
> LLVM just works, as does compiling with the system Clang compiler and
> LLD linker on FreeBSD without any external packages needed (beyond GNU
> make).
> 
> Changes in v4:
>  * Avoid read-only dynamic relocation error with LLD
>  * Support FW_PIC with LLD when using a bare-metal triple
> 
> Changes in v3:
>  * Fixed the binding warning seen when building with LLVM
>  * Provide a default -target when CROSS_COMPILE isn't set so the
>    variable isn't normally required to be set when cross-compiling with
>    LLVM (but is still honoured if set).
>  * Pass -fuse=bfd when not using LLD to ensure an ld.bfd gets correctly
>    picked up over a plain ld when ld is LLD. Now we always explicitly
>    request Clang and GCC use the linker type corresponding to LD. This
>    allows BFD to be used on FreeBSD for testing purposes despite LLD
>    being the system linker.
> 
> Changes in v2:
>  * Add documentation to README.md
>  * Pass -fuse-ld=lld to Clang when using LLD
>  * Modify commit message of first commit to explain -N -> -Wl,-N change
>  * Bring back the old ?= uses for the non-CROSS_COMPILE (and, now,
>    non-LLVM) case; whilst I still think that's how it should be, it's
>    not required for this patch, just related cleanup I did to reduce
>    the complexity of adding the LLVM case.
> 
> Jessica Clarke (5):
>  fw_base: Don't mark fw_platform_init as both global and weak
>  fw_base: Put data in .data rather than .text
>  firmware: Explicitly pass -pie to the linker, not just the driver
>  Makefile: Support building with Clang and LLVM binutils
>  Drop dependency on libgcc by importing part of FreeBSD's libquad
> 
> Makefile                              |  65 +++++-
> README.md                             |  43 +++-
> firmware/fw_base.S                    |   2 +-
> firmware/objects.mk                   |   2 +-
> lib/utils/libquad/divdi3.c            |  64 ++++++
> lib/utils/libquad/include/limits.h    |  12 ++
> lib/utils/libquad/include/sys/cdefs.h |  12 ++
> lib/utils/libquad/include/sys/types.h |  25 +++
> lib/utils/libquad/moddi3.c            |  66 ++++++
> lib/utils/libquad/objects.mk          |  14 ++
> lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
> lib/utils/libquad/quad.h              | 105 ++++++++++
> lib/utils/libquad/udivdi3.c           |  52 +++++
> lib/utils/libquad/umoddi3.c           |  54 +++++
> 14 files changed, 784 insertions(+), 10 deletions(-)
> create mode 100644 lib/utils/libquad/divdi3.c
> create mode 100644 lib/utils/libquad/include/limits.h
> create mode 100644 lib/utils/libquad/include/sys/cdefs.h
> create mode 100644 lib/utils/libquad/include/sys/types.h
> create mode 100644 lib/utils/libquad/moddi3.c
> create mode 100644 lib/utils/libquad/objects.mk
> create mode 100644 lib/utils/libquad/qdivrem.c
> create mode 100644 lib/utils/libquad/quad.h
> create mode 100644 lib/utils/libquad/udivdi3.c
> create mode 100644 lib/utils/libquad/umoddi3.c

... all Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>, managed to
forget that yet again :(

Jess

[PATCH v4 1/5] fw_base: Don't mark fw_platform_init as both global and weak

[Bin Meng]

On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
>
> These are mutually exclusive. GNU as and LLVM both let later binding
> directives override earlier ones so this works as intended, but LLVM 12
> turned this into a warning as there's no good reason to do such a thing
> and could be a potential bug. Thus, remove the redundant and incorrect
> .globl directive for fw_platform_init.
> ---
>  firmware/fw_base.S | 1 -
>  1 file changed, 1 deletion(-)
>

You SoB tag was missed again:

Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>

And please include people's RB / TB tags in the newer version, if
nothing changed:

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

[PATCH v4 2/5] fw_base: Put data in .data rather than .text

[Bin Meng]

On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
>
> The -N linker option is supposed to make .text writable, but GNU ld and
> LLD differ in interpreting what that means. GNU ld will happily let you
> have relocations in it, but LLD will see that the input section is
> read-only (even though the output section is writable) and give an
> error. It's unclear if either of them intend to have that behaviour in
> this edge case, but regardless there's no reason not to just put the
> data in a writable .data section.
> ---
>  firmware/fw_base.S | 1 +
>  1 file changed, 1 insertion(+)
>

Again, your SoB tag was missed:

Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>

With this patch the following error seen when building with full LLVM
toolchain no longer exists:

 ELF       platform/generic/firmware/fw_dynamic.elf
ld.lld: error: can't create dynamic relocation R_RISCV_64 against
symbol: _fw_start in readonly segment; recompile object files with
-fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in opensbi/build/platform/generic/firmware/fw_dynamic.elf.ld:8
>>> referenced by fw_base.S:502 (opensbi/firmware/fw_base.S:502)
>>>               opensbi/build/platform/generic/firmware/fw_dynamic.o:(.entry+0x3A0)

ld.lld: error: can't create dynamic relocation R_RISCV_64 against
symbol: _fw_reloc_end in readonly segment; recompile object files with
-fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in opensbi/build/platform/generic/firmware/fw_dynamic.elf.ld:92
>>> referenced by fw_base.S:502 (opensbi/firmware/fw_base.S:502)
>>>               opensbi/build/platform/generic/firmware/fw_dynamic.o:(.entry+0x3B0)
clang-12: error: linker command failed with exit code 1 (use -v to see
invocation)
make: *** [Makefile:396:
opensbi/build/platform/generic/firmware/fw_dynamic.elf] Error 1

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Bin Meng]

On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
>
> When using Clang with a bare-metal triple, -pie does not get passed to
> the linker as it's not normally a thing that makes sense. However, in
> our case it is, and manually forwarding it on works as desired, so do so
> to fully support FW_PIC with Clang, including when linking with LLD.
> ---
>  firmware/objects.mk | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/firmware/objects.mk b/firmware/objects.mk
> index ce91c2f..3bc83cd 100644
> --- a/firmware/objects.mk
> +++ b/firmware/objects.mk
> @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
>  firmware-genflags-y += -DFW_PIC
>  firmware-asflags-y  += -fpic
>  firmware-cflags-y   += -fPIE -pie
> -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
> +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
>  endif

Does this manual forwarding also work for GNU ld? If so, I think we
don't need to detect bare-metal triple and turn off FW_PIC in
Makefile?

Regards,
Bin

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Bin Meng]

On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> >
> > When using Clang with a bare-metal triple, -pie does not get passed to
> > the linker as it's not normally a thing that makes sense. However, in
> > our case it is, and manually forwarding it on works as desired, so do so
> > to fully support FW_PIC with Clang, including when linking with LLD.
> > ---
> >  firmware/objects.mk | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/firmware/objects.mk b/firmware/objects.mk
> > index ce91c2f..3bc83cd 100644
> > --- a/firmware/objects.mk
> > +++ b/firmware/objects.mk
> > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> >  firmware-genflags-y += -DFW_PIC
> >  firmware-asflags-y  += -fpic
> >  firmware-cflags-y   += -fPIE -pie
> > -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
> > +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
> >  endif
>
> Does this manual forwarding also work for GNU ld? If so, I think we
> don't need to detect bare-metal triple and turn off FW_PIC in
> Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build
the current HEAD of opensbi/master. Indeed it's broken that pie is not
supported with the bare-metal triple.

The GNU ld simply complains:

 ELF       platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

Regards,
Bin

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Anup Patel]

Hi Bin,

On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> > >
> > > When using Clang with a bare-metal triple, -pie does not get passed to
> > > the linker as it's not normally a thing that makes sense. However, in
> > > our case it is, and manually forwarding it on works as desired, so do so
> > > to fully support FW_PIC with Clang, including when linking with LLD.
> > > ---
> > >  firmware/objects.mk | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/firmware/objects.mk b/firmware/objects.mk
> > > index ce91c2f..3bc83cd 100644
> > > --- a/firmware/objects.mk
> > > +++ b/firmware/objects.mk
> > > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> > >  firmware-genflags-y += -DFW_PIC
> > >  firmware-asflags-y  += -fpic
> > >  firmware-cflags-y   += -fPIE -pie
> > > -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
> > > +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
> > >  endif
> >
> > Does this manual forwarding also work for GNU ld? If so, I think we
> > don't need to detect bare-metal triple and turn off FW_PIC in
> > Makefile?
>
> I just built a riscv64-unknown-elf-gcc toolchain and used it to build
> the current HEAD of opensbi/master. Indeed it's broken that pie is not
> supported with the bare-metal triple.
>
> The GNU ld simply complains:
>
>  ELF       platform/generic/firmware/payloads/test.elf
> /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
> -pie not supported

The firmware/objects.mk is doing the following:

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.

We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send it.

Regards,
Anup

>
> Regards,
> Bin
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Bin Meng]

Hi Anup,

On Sat, Jul 10, 2021 at 6:42 PM Anup Patel <anup@brainfault.org> wrote:
>
> Hi Bin,
>
> On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> > > >
> > > > When using Clang with a bare-metal triple, -pie does not get passed to
> > > > the linker as it's not normally a thing that makes sense. However, in
> > > > our case it is, and manually forwarding it on works as desired, so do so
> > > > to fully support FW_PIC with Clang, including when linking with LLD.
> > > > ---
> > > >  firmware/objects.mk | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/firmware/objects.mk b/firmware/objects.mk
> > > > index ce91c2f..3bc83cd 100644
> > > > --- a/firmware/objects.mk
> > > > +++ b/firmware/objects.mk
> > > > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> > > >  firmware-genflags-y += -DFW_PIC
> > > >  firmware-asflags-y  += -fpic
> > > >  firmware-cflags-y   += -fPIE -pie
> > > > -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
> > > > +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
> > > >  endif
> > >
> > > Does this manual forwarding also work for GNU ld? If so, I think we
> > > don't need to detect bare-metal triple and turn off FW_PIC in
> > > Makefile?
> >
> > I just built a riscv64-unknown-elf-gcc toolchain and used it to build
> > the current HEAD of opensbi/master. Indeed it's broken that pie is not
> > supported with the bare-metal triple.
> >
> > The GNU ld simply complains:
> >
> >  ELF       platform/generic/firmware/payloads/test.elf
> > /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
> > -pie not supported
>
> The firmware/objects.mk is doing the following:
>
> ifndef FW_PIC
> FW_PIC := y
> endif
>
> Instead of above, we should set FW_PIC=y only when the underlying
> toolchain supports pie.
>

Agree.

> We need a patch for this to be merged before we can merge this
> series. Can you send such a patch ? If not then I can send it.

Do you have some reliable ways to check whether a toolchain supports PIE?

Regards,
Bin

[PATCH v4 4/5] Makefile: Support building with Clang and LLVM binutils

[Bin Meng]

On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
>
> This is intended to mirror the Linux kernel. Building with CC=clang will
> use Clang as the compiler but default to using the existing binutils.
> Building with LLVM=1 will default to using Clang and LLVM binutils.
>
> Whilst GCC will accept the -N linker option and forward it on to the
> linker, Clang will not, and so in order to support both compilers we
> must use -Wl, to forward it to the linker as is required for most other
> linker options.
>
> Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
> ---
>  Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
>  README.md | 43 ++++++++++++++++++++++++++++++++++--
>  2 files changed, 100 insertions(+), 8 deletions(-)
>

Here are test results:

Building with "riscv64-linux-gcc",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped

$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
...
000080013628  000200000002 R_RISCV_64        0000000080013720
fdt_serial_uart8250 + 0
000080013830  000d00000002 R_RISCV_64        00000000800138d8
fdt_reset_sifive_test + 0

Building with "LLVM=1",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped

$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
...
000080017c90  000000000003 R_RISCV_RELATIVE                     80017628

There are two differences:

1. LLVM toolchain generates a "shared object" firmware image, while
GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries
of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64
entries.

I am not sure whether GCC / LLVM is doing things correctly for the
above 2 differences. fw_dynamic image of both can boot to S-mode
U-Boot on QEMU 'virt' though.

Using clang and GNU binutils, fw_dynamic image does not boot on QEMU
'virt', as reported before.
$ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic

Regards,
Bin

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Anup Patel]

On Sat, Jul 10, 2021 at 6:13 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Anup,
>
> On Sat, Jul 10, 2021 at 6:42 PM Anup Patel <anup@brainfault.org> wrote:
> >
> > Hi Bin,
> >
> > On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> > > > >
> > > > > When using Clang with a bare-metal triple, -pie does not get passed to
> > > > > the linker as it's not normally a thing that makes sense. However, in
> > > > > our case it is, and manually forwarding it on works as desired, so do so
> > > > > to fully support FW_PIC with Clang, including when linking with LLD.
> > > > > ---
> > > > >  firmware/objects.mk | 2 +-
> > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/firmware/objects.mk b/firmware/objects.mk
> > > > > index ce91c2f..3bc83cd 100644
> > > > > --- a/firmware/objects.mk
> > > > > +++ b/firmware/objects.mk
> > > > > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> > > > >  firmware-genflags-y += -DFW_PIC
> > > > >  firmware-asflags-y  += -fpic
> > > > >  firmware-cflags-y   += -fPIE -pie
> > > > > -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
> > > > > +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
> > > > >  endif
> > > >
> > > > Does this manual forwarding also work for GNU ld? If so, I think we
> > > > don't need to detect bare-metal triple and turn off FW_PIC in
> > > > Makefile?
> > >
> > > I just built a riscv64-unknown-elf-gcc toolchain and used it to build
> > > the current HEAD of opensbi/master. Indeed it's broken that pie is not
> > > supported with the bare-metal triple.
> > >
> > > The GNU ld simply complains:
> > >
> > >  ELF       platform/generic/firmware/payloads/test.elf
> > > /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
> > > -pie not supported
> >
> > The firmware/objects.mk is doing the following:
> >
> > ifndef FW_PIC
> > FW_PIC := y
> > endif
> >
> > Instead of above, we should set FW_PIC=y only when the underlying
> > toolchain supports pie.
> >
>
> Agree.
>
> > We need a patch for this to be merged before we can merge this
> > series. Can you send such a patch ? If not then I can send it.
>
> Do you have some reliable ways to check whether a toolchain supports PIE?

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then
we set FW_PIC=y else we set FW_PIC=n. I did not find any info in the
"${CROSS_COMPILE}gcc -v" output.

Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.

Regards,
Anup

>
> Regards,
> Bin

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Bin Meng]

On Sat, Jul 10, 2021 at 9:38 PM Anup Patel <anup@brainfault.org> wrote:
>
> On Sat, Jul 10, 2021 at 6:13 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Anup,
> >
> > On Sat, Jul 10, 2021 at 6:42 PM Anup Patel <anup@brainfault.org> wrote:
> > >
> > > Hi Bin,
> > >
> > > On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> > > > > >
> > > > > > When using Clang with a bare-metal triple, -pie does not get passed to
> > > > > > the linker as it's not normally a thing that makes sense. However, in
> > > > > > our case it is, and manually forwarding it on works as desired, so do so
> > > > > > to fully support FW_PIC with Clang, including when linking with LLD.
> > > > > > ---
> > > > > >  firmware/objects.mk | 2 +-
> > > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/firmware/objects.mk b/firmware/objects.mk
> > > > > > index ce91c2f..3bc83cd 100644
> > > > > > --- a/firmware/objects.mk
> > > > > > +++ b/firmware/objects.mk
> > > > > > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> > > > > >  firmware-genflags-y += -DFW_PIC
> > > > > >  firmware-asflags-y  += -fpic
> > > > > >  firmware-cflags-y   += -fPIE -pie
> > > > > > -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
> > > > > > +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
> > > > > >  endif
> > > > >
> > > > > Does this manual forwarding also work for GNU ld? If so, I think we
> > > > > don't need to detect bare-metal triple and turn off FW_PIC in
> > > > > Makefile?
> > > >
> > > > I just built a riscv64-unknown-elf-gcc toolchain and used it to build
> > > > the current HEAD of opensbi/master. Indeed it's broken that pie is not
> > > > supported with the bare-metal triple.
> > > >
> > > > The GNU ld simply complains:
> > > >
> > > >  ELF       platform/generic/firmware/payloads/test.elf
> > > > /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
> > > > -pie not supported
> > >
> > > The firmware/objects.mk is doing the following:
> > >
> > > ifndef FW_PIC
> > > FW_PIC := y
> > > endif
> > >
> > > Instead of above, we should set FW_PIC=y only when the underlying
> > > toolchain supports pie.
> > >
> >
> > Agree.
> >
> > > We need a patch for this to be merged before we can merge this
> > > series. Can you send such a patch ? If not then I can send it.
> >
> > Do you have some reliable ways to check whether a toolchain supports PIE?
>
> How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then
> we set FW_PIC=y else we set FW_PIC=n.

That works for cross-compile toolchains. But how about native
toolchains (building OpenSBI on a RISC-V machine)?

> I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Me neither.

>
> Maybe also add some comment in objects.mk that GCC bare-metal toolchain
> does not have PIE enabled.

Or we can just document if using bare-metal toolchain FW_PIC has to be
set to n in the build.

Regards,
Bin

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Anup Patel]

On Sat, Jul 10, 2021 at 7:13 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Sat, Jul 10, 2021 at 9:38 PM Anup Patel <anup@brainfault.org> wrote:
> >
> > On Sat, Jul 10, 2021 at 6:13 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > Hi Anup,
> > >
> > > On Sat, Jul 10, 2021 at 6:42 PM Anup Patel <anup@brainfault.org> wrote:
> > > >
> > > > Hi Bin,
> > > >
> > > > On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > >
> > > > > > On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> > > > > > >
> > > > > > > When using Clang with a bare-metal triple, -pie does not get passed to
> > > > > > > the linker as it's not normally a thing that makes sense. However, in
> > > > > > > our case it is, and manually forwarding it on works as desired, so do so
> > > > > > > to fully support FW_PIC with Clang, including when linking with LLD.
> > > > > > > ---
> > > > > > >  firmware/objects.mk | 2 +-
> > > > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > >
> > > > > > > diff --git a/firmware/objects.mk b/firmware/objects.mk
> > > > > > > index ce91c2f..3bc83cd 100644
> > > > > > > --- a/firmware/objects.mk
> > > > > > > +++ b/firmware/objects.mk
> > > > > > > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> > > > > > >  firmware-genflags-y += -DFW_PIC
> > > > > > >  firmware-asflags-y  += -fpic
> > > > > > >  firmware-cflags-y   += -fPIE -pie
> > > > > > > -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
> > > > > > > +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
> > > > > > >  endif
> > > > > >
> > > > > > Does this manual forwarding also work for GNU ld? If so, I think we
> > > > > > don't need to detect bare-metal triple and turn off FW_PIC in
> > > > > > Makefile?
> > > > >
> > > > > I just built a riscv64-unknown-elf-gcc toolchain and used it to build
> > > > > the current HEAD of opensbi/master. Indeed it's broken that pie is not
> > > > > supported with the bare-metal triple.
> > > > >
> > > > > The GNU ld simply complains:
> > > > >
> > > > >  ELF       platform/generic/firmware/payloads/test.elf
> > > > > /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
> > > > > -pie not supported
> > > >
> > > > The firmware/objects.mk is doing the following:
> > > >
> > > > ifndef FW_PIC
> > > > FW_PIC := y
> > > > endif
> > > >
> > > > Instead of above, we should set FW_PIC=y only when the underlying
> > > > toolchain supports pie.
> > > >
> > >
> > > Agree.
> > >
> > > > We need a patch for this to be merged before we can merge this
> > > > series. Can you send such a patch ? If not then I can send it.
> > >
> > > Do you have some reliable ways to check whether a toolchain supports PIE?
> >
> > How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then
> > we set FW_PIC=y else we set FW_PIC=n.
>
> That works for cross-compile toolchains. But how about native
> toolchains (building OpenSBI on a RISC-V machine)?

Yes, it will not work for native compilation of OpenSBI.

>
> > I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
>
> Me neither.
>
> >
> > Maybe also add some comment in objects.mk that GCC bare-metal toolchain
> > does not have PIE enabled.
>
> Or we can just document if using bare-metal toolchain FW_PIC has to be
> set to n in the build.

Sounds good.

We should document it under section "Required Toolchain" of top-level
README.md. Basically, we prefer toolchains with PIE support and for
toolchains not having PIE users have to pass the "FW_PIC=n" option.

Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.

Regards,
Anup

>
> Regards,
> Bin

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Xiang W]

? 2021-07-10???? 20:26 +0530?Anup Patel???
> On Sat, Jul 10, 2021 at 7:13 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > 
> > On Sat, Jul 10, 2021 at 9:38 PM Anup Patel <anup@brainfault.org>
> > wrote:
> > > 
> > > On Sat, Jul 10, 2021 at 6:13 PM Bin Meng <bmeng.cn@gmail.com>
> > > wrote:
> > > > 
> > > > Hi Anup,
> > > > 
> > > > On Sat, Jul 10, 2021 at 6:42 PM Anup Patel
> > > > <anup@brainfault.org> wrote:
> > > > > 
> > > > > Hi Bin,
> > > > > 
> > > > > On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <bmeng.cn@gmail.com>
> > > > > wrote:
> > > > > > 
> > > > > > On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <
> > > > > > bmeng.cn at gmail.com> wrote:
> > > > > > > 
> > > > > > > On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <
> > > > > > > jrtc27 at jrtc27.com> wrote:
> > > > > > > > 
> > > > > > > > When using Clang with a bare-metal triple, -pie does
> > > > > > > > not get passed to
> > > > > > > > the linker as it's not normally a thing that makes
> > > > > > > > sense. However, in
> > > > > > > > our case it is, and manually forwarding it on works as
> > > > > > > > desired, so do so
> > > > > > > > to fully support FW_PIC with Clang, including when
> > > > > > > > linking with LLD.
> > > > > > > > ---
> > > > > > > > ?firmware/objects.mk | 2 +-
> > > > > > > > ?1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > > > 
> > > > > > > > diff --git a/firmware/objects.mk b/firmware/objects.mk
> > > > > > > > index ce91c2f..3bc83cd 100644
> > > > > > > > --- a/firmware/objects.mk
> > > > > > > > +++ b/firmware/objects.mk
> > > > > > > > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> > > > > > > > ?firmware-genflags-y += -DFW_PIC
> > > > > > > > ?firmware-asflags-y? += -fpic
> > > > > > > > ?firmware-cflags-y?? += -fPIE -pie
> > > > > > > > -firmware-ldflags-y? +=? -Wl,--no-dynamic-linker
> > > > > > > > +firmware-ldflags-y? +=? -Wl,--no-dynamic-linker -Wl,-
> > > > > > > > pie
> > > > > > > > ?endif
> > > > > > > 
> > > > > > > Does this manual forwarding also work for GNU ld? If so,
> > > > > > > I think we
> > > > > > > don't need to detect bare-metal triple and turn off
> > > > > > > FW_PIC in
> > > > > > > Makefile?
> > > > > > 
> > > > > > I just built a riscv64-unknown-elf-gcc toolchain and used
> > > > > > it to build
> > > > > > the current HEAD of opensbi/master. Indeed it's broken that
> > > > > > pie is not
> > > > > > supported with the bare-metal triple.
> > > > > > 
> > > > > > The GNU ld simply complains:
> > > > > > 
> > > > > > ?ELF?????? platform/generic/firmware/payloads/test.elf
> > > > > > /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-
> > > > > > elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
> > > > > > -pie not supported
> > > > > 
> > > > > The firmware/objects.mk is doing the following:
> > > > > 
> > > > > ifndef FW_PIC
> > > > > FW_PIC := y
> > > > > endif
> > > > > 
> > > > > Instead of above, we should set FW_PIC=y only when the
> > > > > underlying
> > > > > toolchain supports pie.
> > > > > 
> > > > 
> > > > Agree.
> > > > 
> > > > > We need a patch for this to be merged before we can merge
> > > > > this
> > > > > series. Can you send such a patch ? If not then I can send
> > > > > it.
> > > > 
> > > > Do you have some reliable ways to check whether a toolchain
> > > > supports PIE?
> > > 
> > > How about checking "-linux-" in CROSS_COMPILE prefix ? If it's
> > > available then
> > > we set FW_PIC=y else we set FW_PIC=n.
> > 
> > That works for cross-compile toolchains. But how about native
> > toolchains (building OpenSBI on a RISC-V machine)?
> 
> Yes, it will not work for native compilation of OpenSBI.
> 
> > 
> > > I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
> > 
> > Me neither.
> > 
> > > 
> > > Maybe also add some comment in objects.mk that GCC bare-metal
> > > toolchain
> > > does not have PIE enabled.
> > 
> > Or we can just document if using bare-metal toolchain FW_PIC has to
> > be
> > set to n in the build.
> 
> Sounds good.
> 
> We should document it under section "Required Toolchain" of top-level
> README.md. Basically, we prefer toolchains with PIE support and for
> toolchains not having PIE users have to pass the "FW_PIC=n" option.
> 
> Also, we should replace references of bare-metal toolchain prefix
> with
> linux toolchain prefix everywhere in documentation.
> 
> Regards,
> Anup
We can add the following code to the makefile to detect pie support

pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null
2>/dev/null && echo y || echo n)
> 
> > 
> > Regards,
> > Bin
> 

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Jessica Clarke]

On 10 Jul 2021, at 19:07, Xiang W <wxjstz@126.com> wrote:
> 
> ? 2021-07-10???? 20:26 +0530?Anup Patel???
>> On Sat, Jul 10, 2021 at 7:13 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>>> 
>>> On Sat, Jul 10, 2021 at 9:38 PM Anup Patel <anup@brainfault.org>
>>> wrote:
>>>> 
>>>> On Sat, Jul 10, 2021 at 6:13 PM Bin Meng <bmeng.cn@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hi Anup,
>>>>> 
>>>>> On Sat, Jul 10, 2021 at 6:42 PM Anup Patel
>>>>> <anup@brainfault.org> wrote:
>>>>>> 
>>>>>> Hi Bin,
>>>>>> 
>>>>>> On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <bmeng.cn@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <
>>>>>>> bmeng.cn at gmail.com> wrote:
>>>>>>>> 
>>>>>>>> On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <
>>>>>>>> jrtc27 at jrtc27.com> wrote:
>>>>>>>>> 
>>>>>>>>> When using Clang with a bare-metal triple, -pie does
>>>>>>>>> not get passed to
>>>>>>>>> the linker as it's not normally a thing that makes
>>>>>>>>> sense. However, in
>>>>>>>>> our case it is, and manually forwarding it on works as
>>>>>>>>> desired, so do so
>>>>>>>>> to fully support FW_PIC with Clang, including when
>>>>>>>>> linking with LLD.
>>>>>>>>> ---
>>>>>>>>>  firmware/objects.mk | 2 +-
>>>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>>> 
>>>>>>>>> diff --git a/firmware/objects.mk b/firmware/objects.mk
>>>>>>>>> index ce91c2f..3bc83cd 100644
>>>>>>>>> --- a/firmware/objects.mk
>>>>>>>>> +++ b/firmware/objects.mk
>>>>>>>>> @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
>>>>>>>>>  firmware-genflags-y += -DFW_PIC
>>>>>>>>>  firmware-asflags-y  += -fpic
>>>>>>>>>  firmware-cflags-y   += -fPIE -pie
>>>>>>>>> -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
>>>>>>>>> +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-
>>>>>>>>> pie
>>>>>>>>>  endif
>>>>>>>> 
>>>>>>>> Does this manual forwarding also work for GNU ld? If so,
>>>>>>>> I think we
>>>>>>>> don't need to detect bare-metal triple and turn off
>>>>>>>> FW_PIC in
>>>>>>>> Makefile?
>>>>>>> 
>>>>>>> I just built a riscv64-unknown-elf-gcc toolchain and used
>>>>>>> it to build
>>>>>>> the current HEAD of opensbi/master. Indeed it's broken that
>>>>>>> pie is not
>>>>>>> supported with the bare-metal triple.
>>>>>>> 
>>>>>>> The GNU ld simply complains:
>>>>>>> 
>>>>>>>  ELF       platform/generic/firmware/payloads/test.elf
>>>>>>> /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-
>>>>>>> elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
>>>>>>> -pie not supported
>>>>>> 
>>>>>> The firmware/objects.mk is doing the following:
>>>>>> 
>>>>>> ifndef FW_PIC
>>>>>> FW_PIC := y
>>>>>> endif
>>>>>> 
>>>>>> Instead of above, we should set FW_PIC=y only when the
>>>>>> underlying
>>>>>> toolchain supports pie.
>>>>>> 
>>>>> 
>>>>> Agree.
>>>>> 
>>>>>> We need a patch for this to be merged before we can merge
>>>>>> this
>>>>>> series. Can you send such a patch ? If not then I can send
>>>>>> it.
>>>>> 
>>>>> Do you have some reliable ways to check whether a toolchain
>>>>> supports PIE?
>>>> 
>>>> How about checking "-linux-" in CROSS_COMPILE prefix ? If it's
>>>> available then
>>>> we set FW_PIC=y else we set FW_PIC=n.
>>> 
>>> That works for cross-compile toolchains. But how about native
>>> toolchains (building OpenSBI on a RISC-V machine)?
>> 
>> Yes, it will not work for native compilation of OpenSBI.
>> 
>>> 
>>>> I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
>>> 
>>> Me neither.
>>> 
>>>> 
>>>> Maybe also add some comment in objects.mk that GCC bare-metal
>>>> toolchain
>>>> does not have PIE enabled.
>>> 
>>> Or we can just document if using bare-metal toolchain FW_PIC has to
>>> be
>>> set to n in the build.
>> 
>> Sounds good.
>> 
>> We should document it under section "Required Toolchain" of top-level
>> README.md. Basically, we prefer toolchains with PIE support and for
>> toolchains not having PIE users have to pass the "FW_PIC=n" option.
>> 
>> Also, we should replace references of bare-metal toolchain prefix
>> with
>> linux toolchain prefix everywhere in documentation.
>> 
>> Regards,
>> Anup
> We can add the following code to the makefile to detect pie support
> 
> pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null
> 2>/dev/null && echo y || echo n)

That doesn?t work, /dev/null is not a valid object file so this will
always fail with any toolchain.

Jess

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Xiang W]

? 2021-07-10???? 19:10 +0100?Jessica Clarke???
> On 10 Jul 2021, at 19:07, Xiang W <wxjstz@126.com> wrote:
> > 
> > ? 2021-07-10???? 20:26 +0530?Anup Patel???
> > > On Sat, Jul 10, 2021 at 7:13 PM Bin Meng <bmeng.cn@gmail.com>
> > > wrote:
> > > > 
> > > > On Sat, Jul 10, 2021 at 9:38 PM Anup Patel
> > > > <anup@brainfault.org>
> > > > wrote:
> > > > > 
> > > > > On Sat, Jul 10, 2021 at 6:13 PM Bin Meng <bmeng.cn@gmail.com>
> > > > > wrote:
> > > > > > 
> > > > > > Hi Anup,
> > > > > > 
> > > > > > On Sat, Jul 10, 2021 at 6:42 PM Anup Patel
> > > > > > <anup@brainfault.org> wrote:
> > > > > > > 
> > > > > > > Hi Bin,
> > > > > > > 
> > > > > > > On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <
> > > > > > > bmeng.cn at gmail.com>
> > > > > > > wrote:
> > > > > > > > 
> > > > > > > > On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <
> > > > > > > > bmeng.cn at gmail.com> wrote:
> > > > > > > > > 
> > > > > > > > > On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <
> > > > > > > > > jrtc27 at jrtc27.com> wrote:
> > > > > > > > > > 
> > > > > > > > > > When using Clang with a bare-metal triple, -pie
> > > > > > > > > > does
> > > > > > > > > > not get passed to
> > > > > > > > > > the linker as it's not normally a thing that makes
> > > > > > > > > > sense. However, in
> > > > > > > > > > our case it is, and manually forwarding it on works
> > > > > > > > > > as
> > > > > > > > > > desired, so do so
> > > > > > > > > > to fully support FW_PIC with Clang, including when
> > > > > > > > > > linking with LLD.
> > > > > > > > > > ---
> > > > > > > > > > ?firmware/objects.mk | 2 +-
> > > > > > > > > > ?1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > > > > > 
> > > > > > > > > > diff --git a/firmware/objects.mk
> > > > > > > > > > b/firmware/objects.mk
> > > > > > > > > > index ce91c2f..3bc83cd 100644
> > > > > > > > > > --- a/firmware/objects.mk
> > > > > > > > > > +++ b/firmware/objects.mk
> > > > > > > > > > @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
> > > > > > > > > > ?firmware-genflags-y += -DFW_PIC
> > > > > > > > > > ?firmware-asflags-y? += -fpic
> > > > > > > > > > ?firmware-cflags-y?? += -fPIE -pie
> > > > > > > > > > -firmware-ldflags-y? +=? -Wl,--no-dynamic-linker
> > > > > > > > > > +firmware-ldflags-y? +=? -Wl,--no-dynamic-linker -
> > > > > > > > > > Wl,-
> > > > > > > > > > pie
> > > > > > > > > > ?endif
> > > > > > > > > 
> > > > > > > > > Does this manual forwarding also work for GNU ld? If
> > > > > > > > > so,
> > > > > > > > > I think we
> > > > > > > > > don't need to detect bare-metal triple and turn off
> > > > > > > > > FW_PIC in
> > > > > > > > > Makefile?
> > > > > > > > 
> > > > > > > > I just built a riscv64-unknown-elf-gcc toolchain and
> > > > > > > > used
> > > > > > > > it to build
> > > > > > > > the current HEAD of opensbi/master. Indeed it's broken
> > > > > > > > that
> > > > > > > > pie is not
> > > > > > > > supported with the bare-metal triple.
> > > > > > > > 
> > > > > > > > The GNU ld simply complains:
> > > > > > > > 
> > > > > > > > ?ELF?????? platform/generic/firmware/payloads/test.elf
> > > > > > > > /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-
> > > > > > > > elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
> > > > > > > > -pie not supported
> > > > > > > 
> > > > > > > The firmware/objects.mk is doing the following:
> > > > > > > 
> > > > > > > ifndef FW_PIC
> > > > > > > FW_PIC := y
> > > > > > > endif
> > > > > > > 
> > > > > > > Instead of above, we should set FW_PIC=y only when the
> > > > > > > underlying
> > > > > > > toolchain supports pie.
> > > > > > > 
> > > > > > 
> > > > > > Agree.
> > > > > > 
> > > > > > > We need a patch for this to be merged before we can merge
> > > > > > > this
> > > > > > > series. Can you send such a patch ? If not then I can
> > > > > > > send
> > > > > > > it.
> > > > > > 
> > > > > > Do you have some reliable ways to check whether a toolchain
> > > > > > supports PIE?
> > > > > 
> > > > > How about checking "-linux-" in CROSS_COMPILE prefix ? If
> > > > > it's
> > > > > available then
> > > > > we set FW_PIC=y else we set FW_PIC=n.
> > > > 
> > > > That works for cross-compile toolchains. But how about native
> > > > toolchains (building OpenSBI on a RISC-V machine)?
> > > 
> > > Yes, it will not work for native compilation of OpenSBI.
> > > 
> > > > 
> > > > > I did not find any info in the "${CROSS_COMPILE}gcc -v"
> > > > > output.
> > > > 
> > > > Me neither.
> > > > 
> > > > > 
> > > > > Maybe also add some comment in objects.mk that GCC bare-metal
> > > > > toolchain
> > > > > does not have PIE enabled.
> > > > 
> > > > Or we can just document if using bare-metal toolchain FW_PIC
> > > > has to
> > > > be
> > > > set to n in the build.
> > > 
> > > Sounds good.
> > > 
> > > We should document it under section "Required Toolchain" of top-
> > > level
> > > README.md. Basically, we prefer toolchains with PIE support and
> > > for
> > > toolchains not having PIE users have to pass the "FW_PIC=n"
> > > option.
> > > 
> > > Also, we should replace references of bare-metal toolchain prefix
> > > with
> > > linux toolchain prefix everywhere in documentation.
> > > 
> > > Regards,
> > > Anup
> > We can add the following code to the makefile to detect pie support
> > 
> > pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null
> > 2>/dev/null && echo y || echo n)
> 
> That doesn?t work, /dev/null is not a valid object file so this will
> always fail with any toolchain.
I have tested it, so that the following commands no longer report
errors

CROSS_COMPILE=riscv64-unknown-elf- make PLATFORM=generic

Xiang W
> 
> Jess

[PATCH v4 3/5] firmware: Explicitly pass -pie to the linker, not just the driver

[Jessica Clarke]

On 10 Jul 2021, at 19:27, Xiang W <wxjstz@126.com> wrote:
> 
> ? 2021-07-10???? 19:10 +0100?Jessica Clarke???
>> On 10 Jul 2021, at 19:07, Xiang W <wxjstz@126.com> wrote:
>>> 
>>> ? 2021-07-10???? 20:26 +0530?Anup Patel???
>>>> On Sat, Jul 10, 2021 at 7:13 PM Bin Meng <bmeng.cn@gmail.com>
>>>> wrote:
>>>>> 
>>>>> On Sat, Jul 10, 2021 at 9:38 PM Anup Patel
>>>>> <anup@brainfault.org>
>>>>> wrote:
>>>>>> 
>>>>>> On Sat, Jul 10, 2021 at 6:13 PM Bin Meng <bmeng.cn@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Hi Anup,
>>>>>>> 
>>>>>>> On Sat, Jul 10, 2021 at 6:42 PM Anup Patel
>>>>>>> <anup@brainfault.org> wrote:
>>>>>>>> 
>>>>>>>> Hi Bin,
>>>>>>>> 
>>>>>>>> On Sat, Jul 10, 2021 at 2:23 PM Bin Meng <
>>>>>>>> bmeng.cn at gmail.com>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> On Sat, Jul 10, 2021 at 10:56 AM Bin Meng <
>>>>>>>>> bmeng.cn at gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <
>>>>>>>>>> jrtc27 at jrtc27.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> When using Clang with a bare-metal triple, -pie
>>>>>>>>>>> does
>>>>>>>>>>> not get passed to
>>>>>>>>>>> the linker as it's not normally a thing that makes
>>>>>>>>>>> sense. However, in
>>>>>>>>>>> our case it is, and manually forwarding it on works
>>>>>>>>>>> as
>>>>>>>>>>> desired, so do so
>>>>>>>>>>> to fully support FW_PIC with Clang, including when
>>>>>>>>>>> linking with LLD.
>>>>>>>>>>> ---
>>>>>>>>>>>  firmware/objects.mk | 2 +-
>>>>>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>>>>> 
>>>>>>>>>>> diff --git a/firmware/objects.mk
>>>>>>>>>>> b/firmware/objects.mk
>>>>>>>>>>> index ce91c2f..3bc83cd 100644
>>>>>>>>>>> --- a/firmware/objects.mk
>>>>>>>>>>> +++ b/firmware/objects.mk
>>>>>>>>>>> @@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
>>>>>>>>>>>  firmware-genflags-y += -DFW_PIC
>>>>>>>>>>>  firmware-asflags-y  += -fpic
>>>>>>>>>>>  firmware-cflags-y   += -fPIE -pie
>>>>>>>>>>> -firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
>>>>>>>>>>> +firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -
>>>>>>>>>>> Wl,-
>>>>>>>>>>> pie
>>>>>>>>>>>  endif
>>>>>>>>>> 
>>>>>>>>>> Does this manual forwarding also work for GNU ld? If
>>>>>>>>>> so,
>>>>>>>>>> I think we
>>>>>>>>>> don't need to detect bare-metal triple and turn off
>>>>>>>>>> FW_PIC in
>>>>>>>>>> Makefile?
>>>>>>>>> 
>>>>>>>>> I just built a riscv64-unknown-elf-gcc toolchain and
>>>>>>>>> used
>>>>>>>>> it to build
>>>>>>>>> the current HEAD of opensbi/master. Indeed it's broken
>>>>>>>>> that
>>>>>>>>> pie is not
>>>>>>>>> supported with the bare-metal triple.
>>>>>>>>> 
>>>>>>>>> The GNU ld simply complains:
>>>>>>>>> 
>>>>>>>>>  ELF       platform/generic/firmware/payloads/test.elf
>>>>>>>>> /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-
>>>>>>>>> elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
>>>>>>>>> -pie not supported
>>>>>>>> 
>>>>>>>> The firmware/objects.mk is doing the following:
>>>>>>>> 
>>>>>>>> ifndef FW_PIC
>>>>>>>> FW_PIC := y
>>>>>>>> endif
>>>>>>>> 
>>>>>>>> Instead of above, we should set FW_PIC=y only when the
>>>>>>>> underlying
>>>>>>>> toolchain supports pie.
>>>>>>>> 
>>>>>>> 
>>>>>>> Agree.
>>>>>>> 
>>>>>>>> We need a patch for this to be merged before we can merge
>>>>>>>> this
>>>>>>>> series. Can you send such a patch ? If not then I can
>>>>>>>> send
>>>>>>>> it.
>>>>>>> 
>>>>>>> Do you have some reliable ways to check whether a toolchain
>>>>>>> supports PIE?
>>>>>> 
>>>>>> How about checking "-linux-" in CROSS_COMPILE prefix ? If
>>>>>> it's
>>>>>> available then
>>>>>> we set FW_PIC=y else we set FW_PIC=n.
>>>>> 
>>>>> That works for cross-compile toolchains. But how about native
>>>>> toolchains (building OpenSBI on a RISC-V machine)?
>>>> 
>>>> Yes, it will not work for native compilation of OpenSBI.
>>>> 
>>>>> 
>>>>>> I did not find any info in the "${CROSS_COMPILE}gcc -v"
>>>>>> output.
>>>>> 
>>>>> Me neither.
>>>>> 
>>>>>> 
>>>>>> Maybe also add some comment in objects.mk that GCC bare-metal
>>>>>> toolchain
>>>>>> does not have PIE enabled.
>>>>> 
>>>>> Or we can just document if using bare-metal toolchain FW_PIC
>>>>> has to
>>>>> be
>>>>> set to n in the build.
>>>> 
>>>> Sounds good.
>>>> 
>>>> We should document it under section "Required Toolchain" of top-
>>>> level
>>>> README.md. Basically, we prefer toolchains with PIE support and
>>>> for
>>>> toolchains not having PIE users have to pass the "FW_PIC=n"
>>>> option.
>>>> 
>>>> Also, we should replace references of bare-metal toolchain prefix
>>>> with
>>>> linux toolchain prefix everywhere in documentation.
>>>> 
>>>> Regards,
>>>> Anup
>>> We can add the following code to the makefile to detect pie support
>>> 
>>> pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null
>>> 2>/dev/null && echo y || echo n)
>> 
>> That doesn?t work, /dev/null is not a valid object file so this will
>> always fail with any toolchain.
> I have tested it, so that the following commands no longer report
> errors
> 
> CROSS_COMPILE=riscv64-unknown-elf- make PLATFORM=generic

It doesn?t work with Clang. It will always give an error as it regards
/dev/null as being an object file. You need to force it to parse
/dev/null as a C file with -x c. The same thing also happens with
riscv64-unknown-freebsd12.1-gcc. I don?t know what?s special about
riscv64-unknown-elf-gcc, though that *does* seem to do the right thing
currently, but only that. Even my native x86_64-linux-gnu-gcc treats
/dev/null as an object file. So something weird is going on in GCC
land, but -x c should ensure you always get what you want.

Jess

[PATCH v4 4/5] Makefile: Support building with Clang and LLVM binutils

[Bin Meng]

On Sat, Jul 10, 2021 at 9:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> >
> > This is intended to mirror the Linux kernel. Building with CC=clang will
> > use Clang as the compiler but default to using the existing binutils.
> > Building with LLVM=1 will default to using Clang and LLVM binutils.
> >
> > Whilst GCC will accept the -N linker option and forward it on to the
> > linker, Clang will not, and so in order to support both compilers we
> > must use -Wl, to forward it to the linker as is required for most other
> > linker options.
> >
> > Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
> > ---
> >  Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
> >  README.md | 43 ++++++++++++++++++++++++++++++++++--
> >  2 files changed, 100 insertions(+), 8 deletions(-)
> >
>
> Here are test results:
>
> Building with "riscv64-linux-gcc",
>
> $ file build/platform/generic/firmware/fw_dynamic.elf
> build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
> executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
> debug_info, not stripped
>
> $ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
> Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
> ...
> 000080013628  000200000002 R_RISCV_64        0000000080013720
> fdt_serial_uart8250 + 0
> 000080013830  000d00000002 R_RISCV_64        00000000800138d8
> fdt_reset_sifive_test + 0
>
> Building with "LLVM=1",
>
> $ file build/platform/generic/firmware/fw_dynamic.elf
> build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
> object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
> debug_info, not stripped
>
> $ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
> Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
> 000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
> ...
> 000080017c90  000000000003 R_RISCV_RELATIVE                     80017628
>
> There are two differences:
>
> 1. LLVM toolchain generates a "shared object" firmware image, while
> GCC generates "executable".
> 2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries
> of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64
> entries.

Do you have any explanations on these 2 differences? Are these
possible toolchain bugs?

>
> I am not sure whether GCC / LLVM is doing things correctly for the
> above 2 differences. fw_dynamic image of both can boot to S-mode
> U-Boot on QEMU 'virt' though.
>
> Using clang and GNU binutils, fw_dynamic image does not boot on QEMU
> 'virt', as reported before.
> $ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic

Regards,
Bin

[PATCH v4 4/5] Makefile: Support building with Clang and LLVM binutils

[Jessica Clarke]

On 11 Jul 2021, at 14:53, Bin Meng <bmeng.cn@gmail.com> wrote:
> 
> On Sat, Jul 10, 2021 at 9:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>> 
>> On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
>>> 
>>> This is intended to mirror the Linux kernel. Building with CC=clang will
>>> use Clang as the compiler but default to using the existing binutils.
>>> Building with LLVM=1 will default to using Clang and LLVM binutils.
>>> 
>>> Whilst GCC will accept the -N linker option and forward it on to the
>>> linker, Clang will not, and so in order to support both compilers we
>>> must use -Wl, to forward it to the linker as is required for most other
>>> linker options.
>>> 
>>> Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
>>> ---
>>> Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>> README.md | 43 ++++++++++++++++++++++++++++++++++--
>>> 2 files changed, 100 insertions(+), 8 deletions(-)
>>> 
>> 
>> Here are test results:
>> 
>> Building with "riscv64-linux-gcc",
>> 
>> $ file build/platform/generic/firmware/fw_dynamic.elf
>> build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
>> executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
>> debug_info, not stripped
>> 
>> $ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
>> Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
>>  Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
>> ...
>> 000080013628  000200000002 R_RISCV_64        0000000080013720
>> fdt_serial_uart8250 + 0
>> 000080013830  000d00000002 R_RISCV_64        00000000800138d8
>> fdt_reset_sifive_test + 0
>> 
>> Building with "LLVM=1",
>> 
>> $ file build/platform/generic/firmware/fw_dynamic.elf
>> build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
>> object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
>> debug_info, not stripped
>> 
>> $ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
>> Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
>>  Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
>> 000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
>> ...
>> 000080017c90  000000000003 R_RISCV_RELATIVE                     80017628
>> 
>> There are two differences:
>> 
>> 1. LLVM toolchain generates a "shared object" firmware image, while
>> GCC generates "executable".
>> 2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries
>> of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64
>> entries.
> 
> Do you have any explanations on these 2 differences? Are these
> possible toolchain bugs?

[Hm, I composed this on the 11th but seems I never sent it...]

The first one smells to me like GNU ld is wrong, as executables are
inherently not position-independent, PIEs are always shared objects,
and I?m pretty sure that?s true of the recent -static-pie support too.
In practice for our use cases it doesn?t matter though. It seems this
is yet another undocumented, likely unintended (as it?s a very old
legacy, and mostly unused, option) consequence of using -N/--omagic (in
that it just blindly sets various flags internally and nobody thought
about whether that made sense once -pie was added). Incidentally, I
don?t think we actually need -N/--omagic any more, but that?s a
separate thing.

For the minor difference in number of relocations, that probably just
comes down to minor codegen differences and I wouldn?t worry about it;
with a large enough code base small differences are to be expected.

As for R_RISCV_64, there?s no reason for GNU ld to emit R_RISCV_64
here. It?s technically correct but entirely unnecessary (and can break
legitimate code that assumes only R_RISCV_RELATIVE gets emitted, which
*should* be the case; kernel and run-time linker self-relocation code,
that looks a lot like what OpenSBI is doing here, often likes to assume
that, possibly with R_RISCV_IRELATIVE too if IFUNCs are used). I see it
locally for fdt_serial_uart8250 and fdt_reset_sifive which should in no
way be special, there?s nothing stopping those being evaluated at link
time and leaving R_RISCV_RELATIVE to adjust them at run time as needed
like with all the other symbols.

So I?d regard -N + -pie giving EXEC not DYN, and R_RISCV_64 being
emitted here, as being GNU ld sort-of-bugs, albeit with the former
being extremely ill-defined over what that combination means (beyond
?it does what it does?), with LLD?s output for both being what I would
expect.

Jess

>> I am not sure whether GCC / LLVM is doing things correctly for the
>> above 2 differences. fw_dynamic image of both can boot to S-mode
>> U-Boot on QEMU 'virt' though.
>> 
>> Using clang and GNU binutils, fw_dynamic image does not boot on QEMU
>> 'virt', as reported before.
>> $ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic
> 
> Regards,
> Bin

[PATCH v4 4/5] Makefile: Support building with Clang and LLVM binutils

[Bin Meng]

On Wed, Jul 21, 2021 at 1:49 PM Jessica Clarke <jrtc27@jrtc27.com> wrote:
>
> On 11 Jul 2021, at 14:53, Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > On Sat, Jul 10, 2021 at 9:23 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >>
> >> On Sat, Jul 10, 2021 at 3:35 AM Jessica Clarke <jrtc27@jrtc27.com> wrote:
> >>>
> >>> This is intended to mirror the Linux kernel. Building with CC=clang will
> >>> use Clang as the compiler but default to using the existing binutils.
> >>> Building with LLVM=1 will default to using Clang and LLVM binutils.
> >>>
> >>> Whilst GCC will accept the -N linker option and forward it on to the
> >>> linker, Clang will not, and so in order to support both compilers we
> >>> must use -Wl, to forward it to the linker as is required for most other
> >>> linker options.
> >>>
> >>> Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
> >>> ---
> >>> Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
> >>> README.md | 43 ++++++++++++++++++++++++++++++++++--
> >>> 2 files changed, 100 insertions(+), 8 deletions(-)
> >>>
> >>
> >> Here are test results:
> >>
> >> Building with "riscv64-linux-gcc",
> >>
> >> $ file build/platform/generic/firmware/fw_dynamic.elf
> >> build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
> >> executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
> >> debug_info, not stripped
> >>
> >> $ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
> >> Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
> >>  Offset          Info           Type           Sym. Value    Sym. Name + Addend
> >> 000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
> >> ...
> >> 000080013628  000200000002 R_RISCV_64        0000000080013720
> >> fdt_serial_uart8250 + 0
> >> 000080013830  000d00000002 R_RISCV_64        00000000800138d8
> >> fdt_reset_sifive_test + 0
> >>
> >> Building with "LLVM=1",
> >>
> >> $ file build/platform/generic/firmware/fw_dynamic.elf
> >> build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
> >> object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
> >> debug_info, not stripped
> >>
> >> $ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
> >> Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
> >>  Offset          Info           Type           Sym. Value    Sym. Name + Addend
> >> 000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
> >> 000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
> >> ...
> >> 000080017c90  000000000003 R_RISCV_RELATIVE                     80017628
> >>
> >> There are two differences:
> >>
> >> 1. LLVM toolchain generates a "shared object" firmware image, while
> >> GCC generates "executable".
> >> 2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries
> >> of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64
> >> entries.
> >
> > Do you have any explanations on these 2 differences? Are these
> > possible toolchain bugs?
>
> [Hm, I composed this on the 11th but seems I never sent it...]
>
> The first one smells to me like GNU ld is wrong, as executables are
> inherently not position-independent, PIEs are always shared objects,
> and I?m pretty sure that?s true of the recent -static-pie support too.
> In practice for our use cases it doesn?t matter though. It seems this
> is yet another undocumented, likely unintended (as it?s a very old
> legacy, and mostly unused, option) consequence of using -N/--omagic (in
> that it just blindly sets various flags internally and nobody thought
> about whether that made sense once -pie was added). Incidentally, I
> don?t think we actually need -N/--omagic any more, but that?s a
> separate thing.
>
> For the minor difference in number of relocations, that probably just
> comes down to minor codegen differences and I wouldn?t worry about it;
> with a large enough code base small differences are to be expected.
>
> As for R_RISCV_64, there?s no reason for GNU ld to emit R_RISCV_64
> here. It?s technically correct but entirely unnecessary (and can break
> legitimate code that assumes only R_RISCV_RELATIVE gets emitted, which

Fortunately OpenSBI, as well as U-Boot, handle both R_RISCV_RELATIVE
and R_RISCV_{64,32} here, so they are not broken due to these
unnecessary entries.

> *should* be the case; kernel and run-time linker self-relocation code,
> that looks a lot like what OpenSBI is doing here, often likes to assume
> that, possibly with R_RISCV_IRELATIVE too if IFUNCs are used). I see it
> locally for fdt_serial_uart8250 and fdt_reset_sifive which should in no
> way be special, there?s nothing stopping those being evaluated at link
> time and leaving R_RISCV_RELATIVE to adjust them at run time as needed
> like with all the other symbols.
>
> So I?d regard -N + -pie giving EXEC not DYN, and R_RISCV_64 being
> emitted here, as being GNU ld sort-of-bugs, albeit with the former
> being extremely ill-defined over what that combination means (beyond
> ?it does what it does?), with LLD?s output for both being what I would
> expect.

Thanks a lot for the explanation! Someone can file a defect to the GNU ld :)

Regards,
Bin