[PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

Currently, we place the DTB at 2MB from 4GB or end of DRAM which ever is
lesser. However, Linux kernel can address only 1GB of memory for RV32.
Thus, it can not map anything beyond 3GB (assuming 2GB is the starting address).
As a result, it can not process DT and panic if opensbi dynamic firmware
is used.

Fix this by placing the DTB at 2MB from 3GB or end of DRAM whichever is lower.

Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
 hw/riscv/boot.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
index d62f3dc7581e..9e77b22e4d56 100644
--- a/hw/riscv/boot.c
+++ b/hw/riscv/boot.c
@@ -203,9 +203,9 @@ uint32_t riscv_load_fdt(hwaddr dram_base, uint64_t mem_size, void *fdt)
      * We should put fdt as far as possible to avoid kernel/initrd overwriting
      * its content. But it should be addressable by 32 bit system as well.
      * Thus, put it at an aligned address that less than fdt size from end of
-     * dram or 4GB whichever is lesser.
+     * dram or 3GB whichever is lesser.
      */
-    temp = MIN(dram_end, 4096 * MiB);
+    temp = MIN(dram_end, 3072 * MiB);
     fdt_addr = QEMU_ALIGN_DOWN(temp - fdtsize, 2 * MiB);
 
     fdt_pack(fdt);
-- 
2.25.1

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Palmer Dabbelt]

On Thu, 17 Dec 2020 13:48:26 PST (-0800), Atish Patra wrote:
> Currently, we place the DTB at 2MB from 4GB or end of DRAM which ever is
> lesser. However, Linux kernel can address only 1GB of memory for RV32.
> Thus, it can not map anything beyond 3GB (assuming 2GB is the starting address).
> As a result, it can not process DT and panic if opensbi dynamic firmware
> is used.
>
> Fix this by placing the DTB at 2MB from 3GB or end of DRAM whichever is lower.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
>  hw/riscv/boot.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> index d62f3dc7581e..9e77b22e4d56 100644
> --- a/hw/riscv/boot.c
> +++ b/hw/riscv/boot.c
> @@ -203,9 +203,9 @@ uint32_t riscv_load_fdt(hwaddr dram_base, uint64_t mem_size, void *fdt)
>       * We should put fdt as far as possible to avoid kernel/initrd overwriting
>       * its content. But it should be addressable by 32 bit system as well.
>       * Thus, put it at an aligned address that less than fdt size from end of
> -     * dram or 4GB whichever is lesser.
> +     * dram or 3GB whichever is lesser.
>       */
> -    temp = MIN(dram_end, 4096 * MiB);
> +    temp = MIN(dram_end, 3072 * MiB);
>      fdt_addr = QEMU_ALIGN_DOWN(temp - fdtsize, 2 * MiB);
>
>      fdt_pack(fdt);

Presumably this was the cause of that 32-bit boot issue?

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Thu, 2020-12-17 at 14:31 -0800, Palmer Dabbelt wrote:
> On Thu, 17 Dec 2020 13:48:26 PST (-0800), Atish Patra wrote:
> > Currently, we place the DTB at 2MB from 4GB or end of DRAM which
> > ever is
> > lesser. However, Linux kernel can address only 1GB of memory for
> > RV32.
> > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > starting address).
> > As a result, it can not process DT and panic if opensbi dynamic
> > firmware
> > is used.
> > 
> > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > whichever is lower.
> > 
> > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > ---
> >  hw/riscv/boot.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> > index d62f3dc7581e..9e77b22e4d56 100644
> > --- a/hw/riscv/boot.c
> > +++ b/hw/riscv/boot.c
> > @@ -203,9 +203,9 @@ uint32_t riscv_load_fdt(hwaddr dram_base,
> > uint64_t mem_size, void *fdt)
> >       * We should put fdt as far as possible to avoid kernel/initrd
> > overwriting
> >       * its content. But it should be addressable by 32 bit system
> > as well.
> >       * Thus, put it at an aligned address that less than fdt size
> > from end of
> > -     * dram or 4GB whichever is lesser.
> > +     * dram or 3GB whichever is lesser.
> >       */
> > -    temp = MIN(dram_end, 4096 * MiB);
> > +    temp = MIN(dram_end, 3072 * MiB);
> >      fdt_addr = QEMU_ALIGN_DOWN(temp - fdtsize, 2 * MiB);
> > 
> >      fdt_pack(fdt);
> 
> Presumably this was the cause of that 32-bit boot issue?


This fixes the fw_dynamic case if you use more than 1GB of memory.
The other issue was with the kernel. I sent the kernel fix yesterday.

https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/

Can you include the kernel fix in your next PR ?

-- 
Regards,
Atish

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Palmer Dabbelt]

On Thu, 17 Dec 2020 14:35:10 PST (-0800), Atish Patra wrote:
> On Thu, 2020-12-17 at 14:31 -0800, Palmer Dabbelt wrote:
>> On Thu, 17 Dec 2020 13:48:26 PST (-0800), Atish Patra wrote:
>> > Currently, we place the DTB at 2MB from 4GB or end of DRAM which
>> > ever is
>> > lesser. However, Linux kernel can address only 1GB of memory for
>> > RV32.
>> > Thus, it can not map anything beyond 3GB (assuming 2GB is the
>> > starting address).
>> > As a result, it can not process DT and panic if opensbi dynamic
>> > firmware
>> > is used.
>> > 
>> > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
>> > whichever is lower.
>> > 
>> > Signed-off-by: Atish Patra <atish.patra@wdc.com>
>> > ---
>> >  hw/riscv/boot.c | 4 ++--
>> >  1 file changed, 2 insertions(+), 2 deletions(-)
>> > 
>> > diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
>> > index d62f3dc7581e..9e77b22e4d56 100644
>> > --- a/hw/riscv/boot.c
>> > +++ b/hw/riscv/boot.c
>> > @@ -203,9 +203,9 @@ uint32_t riscv_load_fdt(hwaddr dram_base,
>> > uint64_t mem_size, void *fdt)
>> >       * We should put fdt as far as possible to avoid kernel/initrd
>> > overwriting
>> >       * its content. But it should be addressable by 32 bit system
>> > as well.
>> >       * Thus, put it at an aligned address that less than fdt size
>> > from end of
>> > -     * dram or 4GB whichever is lesser.
>> > +     * dram or 3GB whichever is lesser.
>> >       */
>> > -    temp = MIN(dram_end, 4096 * MiB);
>> > +    temp = MIN(dram_end, 3072 * MiB);
>> >      fdt_addr = QEMU_ALIGN_DOWN(temp - fdtsize, 2 * MiB);
>> > 
>> >      fdt_pack(fdt);
>> 
>> Presumably this was the cause of that 32-bit boot issue?
> 
> 
> This fixes the fw_dynamic case if you use more than 1GB of memory.
> The other issue was with the kernel. I sent the kernel fix yesterday.
> 
> https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> 
> Can you include the kernel fix in your next PR ?

Not the one I'm writing up right now, as I want to get the new stuff up earlier
in the merge window this time around.  I'm planning on sending fixes during the
merge window, though, as we have a few.  We have tomorrow off, so I should be
able to get through all the email and send it either over the weekend or early
next week.

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com> wrote:
>
> Currently, we place the DTB at 2MB from 4GB or end of DRAM which ever is
> lesser. However, Linux kernel can address only 1GB of memory for RV32.
> Thus, it can not map anything beyond 3GB (assuming 2GB is the starting address).
> As a result, it can not process DT and panic if opensbi dynamic firmware
> is used.
>
> Fix this by placing the DTB at 2MB from 3GB or end of DRAM whichever is lower.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
>  hw/riscv/boot.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>

With this patch, 32-bit sifive_u still does not boot kernel with the
following patch applied on 5.10:
https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/

Command I used:
$ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
arch/riscv/boot/Image

32-bit virt cannot boot the same kernel image with memory set to 2G either:
$ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
arch/riscv/boot/Image

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> Hi Atish,
> 
> On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com>
> wrote:
> > 
> > Currently, we place the DTB at 2MB from 4GB or end of DRAM which
> > ever is
> > lesser. However, Linux kernel can address only 1GB of memory for
> > RV32.
> > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > starting address).
> > As a result, it can not process DT and panic if opensbi dynamic
> > firmware
> > is used.
> > 
> > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > whichever is lower.
> > 
> > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > ---
> >  hw/riscv/boot.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> 
> With this patch, 32-bit sifive_u still does not boot kernel with the
> following patch applied on 5.10:
> https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> 
> Command I used:
> $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
> arch/riscv/boot/Image
> 
> 32-bit virt cannot boot the same kernel image with memory set to 2G
> either:
> $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> arch/riscv/boot/Image
> 

Hi Bin,
As mentioned in the email on the linux mailing list, this patch only
solves 2GB problem. sifive_u problem is solved by Alistair's patch[1].

He is planning to send the PR soon. The issue with sifive_u boot was it
was failing the 32 bit test earlier resulting a 2MB aligned address
instead of 4MB.

[1] https://www.mail-archive.com/qemu-devel@nongnu.org/msg767886.html

> Regards,
> Bin

-- 
Regards,
Atish

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > Hi Atish,
> >
> > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com>
> > wrote:
> > >
> > > Currently, we place the DTB at 2MB from 4GB or end of DRAM which
> > > ever is
> > > lesser. However, Linux kernel can address only 1GB of memory for
> > > RV32.
> > > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > > starting address).
> > > As a result, it can not process DT and panic if opensbi dynamic
> > > firmware
> > > is used.
> > >
> > > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > > whichever is lower.
> > >
> > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > ---
> > >  hw/riscv/boot.c | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> >
> > With this patch, 32-bit sifive_u still does not boot kernel with the
> > following patch applied on 5.10:
> > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> >
> > Command I used:
> > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
> > arch/riscv/boot/Image
> >
> > 32-bit virt cannot boot the same kernel image with memory set to 2G
> > either:
> > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > arch/riscv/boot/Image
> >
>
> Hi Bin,
> As mentioned in the email on the linux mailing list, this patch only
> solves 2GB problem. sifive_u problem is solved by Alistair's patch[1].
>
> He is planning to send the PR soon. The issue with sifive_u boot was it
> was failing the 32 bit test earlier resulting a 2MB aligned address
> instead of 4MB.

Ah, I see. However my testing shows that virt with 2G still does not
boot with this patch.

>
> [1] https://www.mail-archive.com/qemu-devel@nongnu.org/msg767886.html
>

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> Hi Atish,
> 
> On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <Atish.Patra@wdc.com>
> wrote:
> > 
> > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > Hi Atish,
> > > 
> > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com>
> > > wrote:
> > > > 
> > > > Currently, we place the DTB at 2MB from 4GB or end of DRAM
> > > > which
> > > > ever is
> > > > lesser. However, Linux kernel can address only 1GB of memory
> > > > for
> > > > RV32.
> > > > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > > > starting address).
> > > > As a result, it can not process DT and panic if opensbi dynamic
> > > > firmware
> > > > is used.
> > > > 
> > > > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > > > whichever is lower.
> > > > 
> > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > ---
> > > >  hw/riscv/boot.c | 4 ++--
> > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > 
> > > 
> > > With this patch, 32-bit sifive_u still does not boot kernel with
> > > the
> > > following patch applied on 5.10:
> > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > 
> > > Command I used:
> > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
> > > arch/riscv/boot/Image
> > > 
> > > 32-bit virt cannot boot the same kernel image with memory set to
> > > 2G
> > > either:
> > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > > arch/riscv/boot/Image
> > > 
> > 
> > Hi Bin,
> > As mentioned in the email on the linux mailing list, this patch
> > only
> > solves 2GB problem. sifive_u problem is solved by Alistair's
> > patch[1].
> > 
> > He is planning to send the PR soon. The issue with sifive_u boot
> > was it
> > was failing the 32 bit test earlier resulting a 2MB aligned address
> > instead of 4MB.
> 
> Ah, I see. However my testing shows that virt with 2G still does not
> boot with this patch.
> 

Strange. I verified again with following combination with -bios and
without bios parameter.

1. virt 32/64 with 1GB/2GB memory
2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's patch included)

Can you share the boot log along with the head commit of Qemu and
commandline ? I am using 5.10 kernel with my kernel fix.

> > 
> > [1] 
> > https://www.mail-archive.com/qemu-devel@nongnu.org/msg767886.html
> > 
> 
> Regards,
> Bin

-- 
Regards,
Atish

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > Hi Atish,
> >
> > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <Atish.Patra@wdc.com>
> > wrote:
> > >
> > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > Hi Atish,
> > > >
> > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com>
> > > > wrote:
> > > > >
> > > > > Currently, we place the DTB at 2MB from 4GB or end of DRAM
> > > > > which
> > > > > ever is
> > > > > lesser. However, Linux kernel can address only 1GB of memory
> > > > > for
> > > > > RV32.
> > > > > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > > > > starting address).
> > > > > As a result, it can not process DT and panic if opensbi dynamic
> > > > > firmware
> > > > > is used.
> > > > >
> > > > > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > > > > whichever is lower.
> > > > >
> > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > ---
> > > > >  hw/riscv/boot.c | 4 ++--
> > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > >
> > > >
> > > > With this patch, 32-bit sifive_u still does not boot kernel with
> > > > the
> > > > following patch applied on 5.10:
> > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > >
> > > > Command I used:
> > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
> > > > arch/riscv/boot/Image
> > > >
> > > > 32-bit virt cannot boot the same kernel image with memory set to
> > > > 2G
> > > > either:
> > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > > > arch/riscv/boot/Image
> > > >
> > >
> > > Hi Bin,
> > > As mentioned in the email on the linux mailing list, this patch
> > > only
> > > solves 2GB problem. sifive_u problem is solved by Alistair's
> > > patch[1].
> > >
> > > He is planning to send the PR soon. The issue with sifive_u boot
> > > was it
> > > was failing the 32 bit test earlier resulting a 2MB aligned address
> > > instead of 4MB.
> >
> > Ah, I see. However my testing shows that virt with 2G still does not
> > boot with this patch.
> >
>
> Strange. I verified again with following combination with -bios and
> without bios parameter.
>
> 1. virt 32/64 with 1GB/2GB memory
> 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's patch included)
>
> Can you share the boot log along with the head commit of Qemu and
> commandline ? I am using 5.10 kernel with my kernel fix.
>

I was using Alistair's QEMU repo for testing and 5.10 kernel with your
kernel fix:

$ git checkout -b testing pull-riscv-to-apply-20201217-1
$ apply this patch
$ mkdir build;cd build;../configure
--target-list=riscv64-softmmu,riscv32-softmmu;make -j

$ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
~/work/git/linux/arch/riscv/boot/Image

OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : riscv-virtio,qemu
Platform Features   : timer,mfdeleg
Platform HART Count : 4
Boot HART ID        : 3
Boot HART ISA       : rv32imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 104 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
PMP0    : 0x80000000-0x8001ffff (A)
PMP1    : 0x00000000-0xffffffff (A,R,W,X)
<hangs here>

$ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5 -kernel
~/work/git/linux/arch/riscv/boot/Image

OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : SiFive HiFive Unleashed A00
Platform Features   : timer,mfdeleg
Platform HART Count : 5
Boot HART ID        : 4
Boot HART ISA       : rv32imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 112 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
PMP0    : 0x80000000-0x8001ffff (A)
PMP1    : 0x00000000-0xffffffff (A,R,W,X)
<hangs here>

The following is sifive_u with 1G:

$ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
~/work/git/linux/arch/riscv/boot/Image

OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : SiFive HiFive Unleashed A00
Platform Features   : timer,mfdeleg
Platform HART Count : 5
Boot HART ID        : 3
Boot HART ISA       : rv32imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 112 KB
Runtime SBI Version : 0.2

MIDELEG : 0x00000222
MEDELEG : 0x0000b109
PMP0    : 0x80000000-0x8001ffff (A)
PMP1    : 0x00000000-0xffffffff (A,R,W,X)
[    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
(bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld (GNU
Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80400000
[    0.000000] efi: UEFI not found.
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000080400000-0x00000000bfffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080400000-0x00000000bfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080400000-0x00000000bfffffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI v0.2 TIME extension detected
[    0.000000] SBI v0.2 IPI extension detected
[    0.000000] SBI v0.2 RFENCE extension detected
[    0.000000] SBI v0.2 HSM extension detected
[    0.000000] CPU with hartid=0 is not available
[    0.000000] CPU with hartid=0 is not available

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Fri, 2020-12-18 at 16:42 +0800, Bin Meng wrote:
> Hi Atish,
> 
> On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <Atish.Patra@wdc.com>
> wrote:
> > 
> > On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > > Hi Atish,
> > > 
> > > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <Atish.Patra@wdc.com>
> > > wrote:
> > > > 
> > > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > > Hi Atish,
> > > > > 
> > > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra
> > > > > <atish.patra@wdc.com>
> > > > > wrote:
> > > > > > 
> > > > > > Currently, we place the DTB at 2MB from 4GB or end of DRAM
> > > > > > which
> > > > > > ever is
> > > > > > lesser. However, Linux kernel can address only 1GB of
> > > > > > memory
> > > > > > for
> > > > > > RV32.
> > > > > > Thus, it can not map anything beyond 3GB (assuming 2GB is
> > > > > > the
> > > > > > starting address).
> > > > > > As a result, it can not process DT and panic if opensbi
> > > > > > dynamic
> > > > > > firmware
> > > > > > is used.
> > > > > > 
> > > > > > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > > > > > whichever is lower.
> > > > > > 
> > > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > > ---
> > > > > >  hw/riscv/boot.c | 4 ++--
> > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > 
> > > > > 
> > > > > With this patch, 32-bit sifive_u still does not boot kernel
> > > > > with
> > > > > the
> > > > > following patch applied on 5.10:
> > > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > > > 
> > > > > Command I used:
> > > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -
> > > > > kernel
> > > > > arch/riscv/boot/Image
> > > > > 
> > > > > 32-bit virt cannot boot the same kernel image with memory set
> > > > > to
> > > > > 2G
> > > > > either:
> > > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > > > > arch/riscv/boot/Image
> > > > > 
> > > > 
> > > > Hi Bin,
> > > > As mentioned in the email on the linux mailing list, this patch
> > > > only
> > > > solves 2GB problem. sifive_u problem is solved by Alistair's
> > > > patch[1].
> > > > 
> > > > He is planning to send the PR soon. The issue with sifive_u
> > > > boot
> > > > was it
> > > > was failing the 32 bit test earlier resulting a 2MB aligned
> > > > address
> > > > instead of 4MB.
> > > 
> > > Ah, I see. However my testing shows that virt with 2G still does
> > > not
> > > boot with this patch.
> > > 
> > 
> > Strange. I verified again with following combination with -bios and
> > without bios parameter.
> > 
> > 1. virt 32/64 with 1GB/2GB memory
> > 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's patch
> > included)
> > 
> > Can you share the boot log along with the head commit of Qemu and
> > commandline ? I am using 5.10 kernel with my kernel fix.
> > 
> 
> I was using Alistair's QEMU repo for testing and 5.10 kernel with
> your
> kernel fix:
> 
> $ git checkout -b testing pull-riscv-to-apply-20201217-1
> $ apply this patch
> $ mkdir build;cd build;../configure
> --target-list=riscv64-softmmu,riscv32-softmmu;make -j
> 
> $ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> ~/work/git/linux/arch/riscv/boot/Image
> 
> OpenSBI v0.8
>    ____                    _____ ____ _____
>   / __ \                  / ____|  _ \_   _|
>  | |  | |_ __   ___ _ __ | (___ | |_) || |
>  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>  | |__| | |_) |  __/ | | |____) | |_) || |_
>   \____/| .__/ \___|_| |_|_____/|____/_____|
>         | |
>         |_|
> 
> Platform Name       : riscv-virtio,qemu
> Platform Features   : timer,mfdeleg
> Platform HART Count : 4
> Boot HART ID        : 3
> Boot HART ISA       : rv32imafdcsu
> BOOT HART Features  : pmp,scounteren,mcounteren,time
> BOOT HART PMP Count : 16
> Firmware Base       : 0x80000000
> Firmware Size       : 104 KB
> Runtime SBI Version : 0.2
> 
> MIDELEG : 0x00000222
> MEDELEG : 0x0000b109
> PMP0    : 0x80000000-0x8001ffff (A)
> PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> <hangs here>
> 
> $ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5 -kernel
> ~/work/git/linux/arch/riscv/boot/Image
> 
> OpenSBI v0.8
>    ____                    _____ ____ _____
>   / __ \                  / ____|  _ \_   _|
>  | |  | |_ __   ___ _ __ | (___ | |_) || |
>  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>  | |__| | |_) |  __/ | | |____) | |_) || |_
>   \____/| .__/ \___|_| |_|_____/|____/_____|
>         | |
>         |_|
> 
> Platform Name       : SiFive HiFive Unleashed A00
> Platform Features   : timer,mfdeleg
> Platform HART Count : 5
> Boot HART ID        : 4
> Boot HART ISA       : rv32imafdcsu
> BOOT HART Features  : pmp,scounteren,mcounteren
> BOOT HART PMP Count : 16
> Firmware Base       : 0x80000000
> Firmware Size       : 112 KB
> Runtime SBI Version : 0.2
> 
> MIDELEG : 0x00000222
> MEDELEG : 0x0000b109
> PMP0    : 0x80000000-0x8001ffff (A)
> PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> <hangs here>
> 
> The following is sifive_u with 1G:
> 
> $ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
> ~/work/git/linux/arch/riscv/boot/Image
> 
> OpenSBI v0.8
>    ____                    _____ ____ _____
>   / __ \                  / ____|  _ \_   _|
>  | |  | |_ __   ___ _ __ | (___ | |_) || |
>  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>  | |__| | |_) |  __/ | | |____) | |_) || |_
>   \____/| .__/ \___|_| |_|_____/|____/_____|
>         | |
>         |_|
> 
> Platform Name       : SiFive HiFive Unleashed A00
> Platform Features   : timer,mfdeleg
> Platform HART Count : 5
> Boot HART ID        : 3
> Boot HART ISA       : rv32imafdcsu
> BOOT HART Features  : pmp,scounteren,mcounteren
> BOOT HART PMP Count : 16
> Firmware Base       : 0x80000000
> Firmware Size       : 112 KB
> Runtime SBI Version : 0.2
> 
> MIDELEG : 0x00000222
> MEDELEG : 0x0000b109
> PMP0    : 0x80000000-0x8001ffff (A)
> PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> [    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
> (bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld (GNU
> Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
> [    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80400000
> [    0.000000] efi: UEFI not found.
> [    0.000000] Zone ranges:
> [    0.000000]   Normal   [mem 0x0000000080400000-0x00000000bfffffff]
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000080400000-
> 0x00000000bfffffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000080400000-
> 0x00000000bfffffff]
> [    0.000000] SBI specification v0.2 detected
> [    0.000000] SBI implementation ID=0x1 Version=0x8
> [    0.000000] SBI v0.2 TIME extension detected
> [    0.000000] SBI v0.2 IPI extension detected
> [    0.000000] SBI v0.2 RFENCE extension detected
> [    0.000000] SBI v0.2 HSM extension detected
> [    0.000000] CPU with hartid=0 is not available
> [    0.000000] CPU with hartid=0 is not available
> 
> Regards,
> Bin

May be you forgot to apply this patch for 2GB case on top of Alistair's
tree? I don't see any issues with exact same setup.

Qemu git log
------
851966c92cf5 (HEAD) RISC-V: Place DTB at 3GB boundary instead of 4GB
d31e970a01e7 (tag: pull-riscv-to-apply-20201217-1, alistair/riscv-to-
apply.next, alistair/riscv-to-apply.for-upstream) riscv/opentitan:
Update the OpenTitan memory layout
3ed2b8ac2dac hw/riscv: Use the CPU to determine if 32-bit
094b072c6819 target/riscv: cpu: Set XLEN independently from target
-------

Virt machine boot with 2GB memory
---------------------------------
$ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
~/workspace/linux/arch/riscv/boot/Image                               
                                                                      
OpenSBI v0.8                                                          
   ____                    _____ ____ _____                           
  / __ \                  / ____|  _ \_   _|                          
 | |  | |_ __   ___ _ __ | (___ | |_) || |                            
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |                            
 | |__| | |_) |  __/ | | |____) | |_) || |_                           
  \____/| .__/ \___|_| |_|_____/|____/_____|                          
        | |                                                           
        |_|                                                           
                                                                      
Platform Name       : riscv-virtio,qemu                               
Platform Features   : timer,mfdeleg                                   
Platform HART Count : 4                                               
Boot HART ID        : 0                                               
Boot HART ISA       : rv32imafdcsu                                    
BOOT HART Features  : pmp,scounteren,mcounteren,time                  
BOOT HART PMP Count : 16                                              
Firmware Base       : 0x80000000                                      
Firmware Size       : 104 KB                                          
Runtime SBI Version : 0.2                                             
                                                                      
MIDELEG : 0x00000222                                                  
MEDELEG : 0x0000b109                                                  
PMP0    : 0x80000000-0x8001ffff (A)                                   
PMP1    : 0x00000000-0xffffffff (A,R,W,X)                             
[    0.000000] Linux version 5.10.0-00022-ge20097fb37e2-dirty
(atish@jedi-01) (riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0, GNU ld
(GNU Binutils) 2.35) #548 SMP PREEMPT Fri Dec 18 10:45:11 PST 2020    
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80400000 
[    0.000000] efi: UEFI not found.                                   
[    0.000000] Zone ranges:                                           
[    0.000000]   Normal   [mem 0x0000000080400000-0x00000000c03fffff] 
[    0.000000] Movable zone start for each node                       
[    0.000000] Early memory node ranges                               
[    0.000000]   node   0: [mem 0x0000000080400000-0x00000000c03fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080400000-
0x00000000c03fffff]                                                   
[    0.000000] On node 0 totalpages: 262144                           
[    0.000000]   Normal zone: 2048 pages used for memmap              
[    0.000000]   Normal zone: 0 pages reserved                        
[    0.000000]   Normal zone: 262144 pages, LIFO batch:63   
---------------------------------


sifive_u boot with 2GB memory
---------------------------------
$ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5 -kernel
~/workspace/linux/arch/riscv/boot/Image                               
                                                                      
OpenSBI v0.8                                                          
   ____                    _____ ____ _____                           
  / __ \                  / ____|  _ \_   _|                          
 | |  | |_ __   ___ _ __ | (___ | |_) || |                            
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |                            
 | |__| | |_) |  __/ | | |____) | |_) || |_                           
  \____/| .__/ \___|_| |_|_____/|____/_____|                          
        | |                                                           
        |_|                                                           
                                                                      
Platform Name       : SiFive HiFive Unleashed A00                     
Platform Features   : timer,mfdeleg                                   
Platform HART Count : 5                                               
Boot HART ID        : 3                                               
Boot HART ISA       : rv32imafdcsu                                    
BOOT HART Features  : pmp,scounteren,mcounteren                       
BOOT HART PMP Count : 16                                              
Firmware Base       : 0x80000000                                      
Firmware Size       : 112 KB                                          
Runtime SBI Version : 0.2                                             
                                                                      
MIDELEG : 0x00000222                                                  
MEDELEG : 0x0000b109                                                  
PMP0    : 0x80000000-0x8001ffff (A)                                   
PMP1    : 0x00000000-0xffffffff (A,R,W,X)                             
[    0.000000] Linux version 5.10.0-00022-ge20097fb37e2-dirty
(atish@jedi-01) (riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0, GNU ld
(GNU Binutils) 2.35) #548 SMP PREEMPT Fri Dec 18 10:45:11 PST 2020    
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80400000 
[    0.000000] efi: UEFI not found.                                   
[    0.000000] Zone ranges:                                           
[    0.000000]   Normal   [mem 0x0000000080400000-0x00000000c03fffff] 
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080400000-0x00000000c03fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080400000-
0x00000000c03fffff]

---------------------------------






-- 
Regards,
Atish

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Sat, Dec 19, 2020 at 3:46 AM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Fri, 2020-12-18 at 16:42 +0800, Bin Meng wrote:
> > Hi Atish,
> >
> > On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <Atish.Patra@wdc.com>
> > wrote:
> > >
> > > On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > > > Hi Atish,
> > > >
> > > > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <Atish.Patra@wdc.com>
> > > > wrote:
> > > > >
> > > > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > > > Hi Atish,
> > > > > >
> > > > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra
> > > > > > <atish.patra@wdc.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Currently, we place the DTB at 2MB from 4GB or end of DRAM
> > > > > > > which
> > > > > > > ever is
> > > > > > > lesser. However, Linux kernel can address only 1GB of
> > > > > > > memory
> > > > > > > for
> > > > > > > RV32.
> > > > > > > Thus, it can not map anything beyond 3GB (assuming 2GB is
> > > > > > > the
> > > > > > > starting address).
> > > > > > > As a result, it can not process DT and panic if opensbi
> > > > > > > dynamic
> > > > > > > firmware
> > > > > > > is used.
> > > > > > >
> > > > > > > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > > > > > > whichever is lower.
> > > > > > >
> > > > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > > > ---
> > > > > > >  hw/riscv/boot.c | 4 ++--
> > > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > >
> > > > > >
> > > > > > With this patch, 32-bit sifive_u still does not boot kernel
> > > > > > with
> > > > > > the
> > > > > > following patch applied on 5.10:
> > > > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > > > >
> > > > > > Command I used:
> > > > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -
> > > > > > kernel
> > > > > > arch/riscv/boot/Image
> > > > > >
> > > > > > 32-bit virt cannot boot the same kernel image with memory set
> > > > > > to
> > > > > > 2G
> > > > > > either:
> > > > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > > > > > arch/riscv/boot/Image
> > > > > >
> > > > >
> > > > > Hi Bin,
> > > > > As mentioned in the email on the linux mailing list, this patch
> > > > > only
> > > > > solves 2GB problem. sifive_u problem is solved by Alistair's
> > > > > patch[1].
> > > > >
> > > > > He is planning to send the PR soon. The issue with sifive_u
> > > > > boot
> > > > > was it
> > > > > was failing the 32 bit test earlier resulting a 2MB aligned
> > > > > address
> > > > > instead of 4MB.
> > > >
> > > > Ah, I see. However my testing shows that virt with 2G still does
> > > > not
> > > > boot with this patch.
> > > >
> > >
> > > Strange. I verified again with following combination with -bios and
> > > without bios parameter.
> > >
> > > 1. virt 32/64 with 1GB/2GB memory
> > > 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's patch
> > > included)
> > >
> > > Can you share the boot log along with the head commit of Qemu and
> > > commandline ? I am using 5.10 kernel with my kernel fix.
> > >
> >
> > I was using Alistair's QEMU repo for testing and 5.10 kernel with
> > your
> > kernel fix:
> >
> > $ git checkout -b testing pull-riscv-to-apply-20201217-1
> > $ apply this patch
> > $ mkdir build;cd build;../configure
> > --target-list=riscv64-softmmu,riscv32-softmmu;make -j
> >
> > $ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > ~/work/git/linux/arch/riscv/boot/Image
> >
> > OpenSBI v0.8
> >    ____                    _____ ____ _____
> >   / __ \                  / ____|  _ \_   _|
> >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> >  | |__| | |_) |  __/ | | |____) | |_) || |_
> >   \____/| .__/ \___|_| |_|_____/|____/_____|
> >         | |
> >         |_|
> >
> > Platform Name       : riscv-virtio,qemu
> > Platform Features   : timer,mfdeleg
> > Platform HART Count : 4
> > Boot HART ID        : 3
> > Boot HART ISA       : rv32imafdcsu
> > BOOT HART Features  : pmp,scounteren,mcounteren,time
> > BOOT HART PMP Count : 16
> > Firmware Base       : 0x80000000
> > Firmware Size       : 104 KB
> > Runtime SBI Version : 0.2
> >
> > MIDELEG : 0x00000222
> > MEDELEG : 0x0000b109
> > PMP0    : 0x80000000-0x8001ffff (A)
> > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > <hangs here>
> >
> > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5 -kernel
> > ~/work/git/linux/arch/riscv/boot/Image
> >
> > OpenSBI v0.8
> >    ____                    _____ ____ _____
> >   / __ \                  / ____|  _ \_   _|
> >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> >  | |__| | |_) |  __/ | | |____) | |_) || |_
> >   \____/| .__/ \___|_| |_|_____/|____/_____|
> >         | |
> >         |_|
> >
> > Platform Name       : SiFive HiFive Unleashed A00
> > Platform Features   : timer,mfdeleg
> > Platform HART Count : 5
> > Boot HART ID        : 4
> > Boot HART ISA       : rv32imafdcsu
> > BOOT HART Features  : pmp,scounteren,mcounteren
> > BOOT HART PMP Count : 16
> > Firmware Base       : 0x80000000
> > Firmware Size       : 112 KB
> > Runtime SBI Version : 0.2
> >
> > MIDELEG : 0x00000222
> > MEDELEG : 0x0000b109
> > PMP0    : 0x80000000-0x8001ffff (A)
> > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > <hangs here>
> >
> > The following is sifive_u with 1G:
> >
> > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -kernel
> > ~/work/git/linux/arch/riscv/boot/Image
> >
> > OpenSBI v0.8
> >    ____                    _____ ____ _____
> >   / __ \                  / ____|  _ \_   _|
> >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> >  | |__| | |_) |  __/ | | |____) | |_) || |_
> >   \____/| .__/ \___|_| |_|_____/|____/_____|
> >         | |
> >         |_|
> >
> > Platform Name       : SiFive HiFive Unleashed A00
> > Platform Features   : timer,mfdeleg
> > Platform HART Count : 5
> > Boot HART ID        : 3
> > Boot HART ISA       : rv32imafdcsu
> > BOOT HART Features  : pmp,scounteren,mcounteren
> > BOOT HART PMP Count : 16
> > Firmware Base       : 0x80000000
> > Firmware Size       : 112 KB
> > Runtime SBI Version : 0.2
> >
> > MIDELEG : 0x00000222
> > MEDELEG : 0x0000b109
> > PMP0    : 0x80000000-0x8001ffff (A)
> > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > [    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
> > (bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld (GNU
> > Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
> > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80400000
> > [    0.000000] efi: UEFI not found.
> > [    0.000000] Zone ranges:
> > [    0.000000]   Normal   [mem 0x0000000080400000-0x00000000bfffffff]
> > [    0.000000] Movable zone start for each node
> > [    0.000000] Early memory node ranges
> > [    0.000000]   node   0: [mem 0x0000000080400000-
> > 0x00000000bfffffff]
> > [    0.000000] Initmem setup node 0 [mem 0x0000000080400000-
> > 0x00000000bfffffff]
> > [    0.000000] SBI specification v0.2 detected
> > [    0.000000] SBI implementation ID=0x1 Version=0x8
> > [    0.000000] SBI v0.2 TIME extension detected
> > [    0.000000] SBI v0.2 IPI extension detected
> > [    0.000000] SBI v0.2 RFENCE extension detected
> > [    0.000000] SBI v0.2 HSM extension detected
> > [    0.000000] CPU with hartid=0 is not available
> > [    0.000000] CPU with hartid=0 is not available
> >
> > Regards,
> > Bin
>
> May be you forgot to apply this patch for 2GB case on top of Alistair's
> tree? I don't see any issues with exact same setup.

That's really weird. I have:

$ git log --oneline
3ced2fb RISC-V: Place DTB at 3GB boundary instead of 4GB
d31e970 riscv/opentitan: Update the OpenTitan memory layout
3ed2b8a hw/riscv: Use the CPU to determine if 32-bit
094b072 target/riscv: cpu: Set XLEN independently from target
8987cdc4 target/riscv: csr: Remove compile time XLEN checks

I just rebuilt the QEMU binaries but still have the same result. 2G
does not boot on either 'virt' or 'sifive_u'.

$ ./qemu-system-riscv32 -version
QEMU emulator version 5.2.50 (v5.2.0-551-g3ced2fb)
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers

Maybe it's toolchain related? I am using kernel.org toolchain
riscv64-linux-gcc (GCC) 8.1.0. I see you were using
riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0.

>
> Qemu git log
> ------
> 851966c92cf5 (HEAD) RISC-V: Place DTB at 3GB boundary instead of 4GB
> d31e970a01e7 (tag: pull-riscv-to-apply-20201217-1, alistair/riscv-to-
> apply.next, alistair/riscv-to-apply.for-upstream) riscv/opentitan:
> Update the OpenTitan memory layout
> 3ed2b8ac2dac hw/riscv: Use the CPU to determine if 32-bit
> 094b072c6819 target/riscv: cpu: Set XLEN independently from target
> -------

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Tue, 2020-12-22 at 13:35 +0800, Bin Meng wrote:
> Hi Atish,
> 
> On Sat, Dec 19, 2020 at 3:46 AM Atish Patra <Atish.Patra@wdc.com>
> wrote:
> > 
> > On Fri, 2020-12-18 at 16:42 +0800, Bin Meng wrote:
> > > Hi Atish,
> > > 
> > > On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <Atish.Patra@wdc.com>
> > > wrote:
> > > > 
> > > > On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > > > > Hi Atish,
> > > > > 
> > > > > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra < 
> > > > > Atish.Patra@wdc.com>
> > > > > wrote:
> > > > > > 
> > > > > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > > > > Hi Atish,
> > > > > > > 
> > > > > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra
> > > > > > > <atish.patra@wdc.com>
> > > > > > > wrote:
> > > > > > > > 
> > > > > > > > Currently, we place the DTB at 2MB from 4GB or end of
> > > > > > > > DRAM
> > > > > > > > which
> > > > > > > > ever is
> > > > > > > > lesser. However, Linux kernel can address only 1GB of
> > > > > > > > memory
> > > > > > > > for
> > > > > > > > RV32.
> > > > > > > > Thus, it can not map anything beyond 3GB (assuming 2GB
> > > > > > > > is
> > > > > > > > the
> > > > > > > > starting address).
> > > > > > > > As a result, it can not process DT and panic if opensbi
> > > > > > > > dynamic
> > > > > > > > firmware
> > > > > > > > is used.
> > > > > > > > 
> > > > > > > > Fix this by placing the DTB at 2MB from 3GB or end of
> > > > > > > > DRAM
> > > > > > > > whichever is lower.
> > > > > > > > 
> > > > > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > > > > ---
> > > > > > > >  hw/riscv/boot.c | 4 ++--
> > > > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > > > 
> > > > > > > 
> > > > > > > With this patch, 32-bit sifive_u still does not boot
> > > > > > > kernel
> > > > > > > with
> > > > > > > the
> > > > > > > following patch applied on 5.10:
> > > > > > >  
> > > > > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > > > > > 
> > > > > > > Command I used:
> > > > > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5
> > > > > > > -
> > > > > > > kernel
> > > > > > > arch/riscv/boot/Image
> > > > > > > 
> > > > > > > 32-bit virt cannot boot the same kernel image with memory
> > > > > > > set
> > > > > > > to
> > > > > > > 2G
> > > > > > > either:
> > > > > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -
> > > > > > > kernel
> > > > > > > arch/riscv/boot/Image
> > > > > > > 
> > > > > > 
> > > > > > Hi Bin,
> > > > > > As mentioned in the email on the linux mailing list, this
> > > > > > patch
> > > > > > only
> > > > > > solves 2GB problem. sifive_u problem is solved by
> > > > > > Alistair's
> > > > > > patch[1].
> > > > > > 
> > > > > > He is planning to send the PR soon. The issue with sifive_u
> > > > > > boot
> > > > > > was it
> > > > > > was failing the 32 bit test earlier resulting a 2MB aligned
> > > > > > address
> > > > > > instead of 4MB.
> > > > > 
> > > > > Ah, I see. However my testing shows that virt with 2G still
> > > > > does
> > > > > not
> > > > > boot with this patch.
> > > > > 
> > > > 
> > > > Strange. I verified again with following combination with -bios
> > > > and
> > > > without bios parameter.
> > > > 
> > > > 1. virt 32/64 with 1GB/2GB memory
> > > > 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's patch
> > > > included)
> > > > 
> > > > Can you share the boot log along with the head commit of Qemu
> > > > and
> > > > commandline ? I am using 5.10 kernel with my kernel fix.
> > > > 
> > > 
> > > I was using Alistair's QEMU repo for testing and 5.10 kernel with
> > > your
> > > kernel fix:
> > > 
> > > $ git checkout -b testing pull-riscv-to-apply-20201217-1
> > > $ apply this patch
> > > $ mkdir build;cd build;../configure
> > > --target-list=riscv64-softmmu,riscv32-softmmu;make -j
> > > 
> > > $ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > > ~/work/git/linux/arch/riscv/boot/Image
> > > 
> > > OpenSBI v0.8
> > >    ____                    _____ ____ _____
> > >   / __ \                  / ____|  _ \_   _|
> > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > >         | |
> > >         |_|
> > > 
> > > Platform Name       : riscv-virtio,qemu
> > > Platform Features   : timer,mfdeleg
> > > Platform HART Count : 4
> > > Boot HART ID        : 3
> > > Boot HART ISA       : rv32imafdcsu
> > > BOOT HART Features  : pmp,scounteren,mcounteren,time
> > > BOOT HART PMP Count : 16
> > > Firmware Base       : 0x80000000
> > > Firmware Size       : 104 KB
> > > Runtime SBI Version : 0.2
> > > 
> > > MIDELEG : 0x00000222
> > > MEDELEG : 0x0000b109
> > > PMP0    : 0x80000000-0x8001ffff (A)
> > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > <hangs here>
> > > 
> > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5 -
> > > kernel
> > > ~/work/git/linux/arch/riscv/boot/Image
> > > 
> > > OpenSBI v0.8
> > >    ____                    _____ ____ _____
> > >   / __ \                  / ____|  _ \_   _|
> > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > >         | |
> > >         |_|
> > > 
> > > Platform Name       : SiFive HiFive Unleashed A00
> > > Platform Features   : timer,mfdeleg
> > > Platform HART Count : 5
> > > Boot HART ID        : 4
> > > Boot HART ISA       : rv32imafdcsu
> > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > BOOT HART PMP Count : 16
> > > Firmware Base       : 0x80000000
> > > Firmware Size       : 112 KB
> > > Runtime SBI Version : 0.2
> > > 
> > > MIDELEG : 0x00000222
> > > MEDELEG : 0x0000b109
> > > PMP0    : 0x80000000-0x8001ffff (A)
> > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > <hangs here>
> > > 
> > > The following is sifive_u with 1G:
> > > 
> > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -
> > > kernel
> > > ~/work/git/linux/arch/riscv/boot/Image
> > > 
> > > OpenSBI v0.8
> > >    ____                    _____ ____ _____
> > >   / __ \                  / ____|  _ \_   _|
> > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > >         | |
> > >         |_|
> > > 
> > > Platform Name       : SiFive HiFive Unleashed A00
> > > Platform Features   : timer,mfdeleg
> > > Platform HART Count : 5
> > > Boot HART ID        : 3
> > > Boot HART ISA       : rv32imafdcsu
> > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > BOOT HART PMP Count : 16
> > > Firmware Base       : 0x80000000
> > > Firmware Size       : 112 KB
> > > Runtime SBI Version : 0.2
> > > 
> > > MIDELEG : 0x00000222
> > > MEDELEG : 0x0000b109
> > > PMP0    : 0x80000000-0x8001ffff (A)
> > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > [    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
> > > (bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld (GNU
> > > Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
> > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 -
> > > 0x80400000
> > > [    0.000000] efi: UEFI not found.
> > > [    0.000000] Zone ranges:
> > > [    0.000000]   Normal   [mem 0x0000000080400000-
> > > 0x00000000bfffffff]
> > > [    0.000000] Movable zone start for each node
> > > [    0.000000] Early memory node ranges
> > > [    0.000000]   node   0: [mem 0x0000000080400000-
> > > 0x00000000bfffffff]
> > > [    0.000000] Initmem setup node 0 [mem 0x0000000080400000-
> > > 0x00000000bfffffff]
> > > [    0.000000] SBI specification v0.2 detected
> > > [    0.000000] SBI implementation ID=0x1 Version=0x8
> > > [    0.000000] SBI v0.2 TIME extension detected
> > > [    0.000000] SBI v0.2 IPI extension detected
> > > [    0.000000] SBI v0.2 RFENCE extension detected
> > > [    0.000000] SBI v0.2 HSM extension detected
> > > [    0.000000] CPU with hartid=0 is not available
> > > [    0.000000] CPU with hartid=0 is not available
> > > 
> > > Regards,
> > > Bin
> > 
> > May be you forgot to apply this patch for 2GB case on top of
> > Alistair's
> > tree? I don't see any issues with exact same setup.
> 
> That's really weird. I have:
> 
> $ git log --oneline
> 3ced2fb RISC-V: Place DTB at 3GB boundary instead of 4GB
> d31e970 riscv/opentitan: Update the OpenTitan memory layout
> 3ed2b8a hw/riscv: Use the CPU to determine if 32-bit
> 094b072 target/riscv: cpu: Set XLEN independently from target
> 8987cdc4 target/riscv: csr: Remove compile time XLEN checks
> 
> I just rebuilt the QEMU binaries but still have the same result. 2G
> does not boot on either 'virt' or 'sifive_u'.
> 
> $ ./qemu-system-riscv32 -version
> QEMU emulator version 5.2.50 (v5.2.0-551-g3ced2fb)
> Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project
> developers
> 
> Maybe it's toolchain related? I am using kernel.org toolchain
> riscv64-linux-gcc (GCC) 8.1.0. I see you were using
> riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0.
> 

Strange. If it is a toolchain related issue, the cause might be
completely different. Can you give it a try to with updated toolchain ?
The pre-built toolchain available in bootlin[1] is 10.2.0.

[1] https://toolchains.bootlin.com/

Can I download pre-built gcc 8.1.0 toolchain from somewhere ? 

> 

> > 
> > Qemu git log
> > ------
> > 851966c92cf5 (HEAD) RISC-V: Place DTB at 3GB boundary instead of
> > 4GB
> > d31e970a01e7 (tag: pull-riscv-to-apply-20201217-1, alistair/riscv-
> > to-
> > apply.next, alistair/riscv-to-apply.for-upstream) riscv/opentitan:
> > Update the OpenTitan memory layout
> > 3ed2b8ac2dac hw/riscv: Use the CPU to determine if 32-bit
> > 094b072c6819 target/riscv: cpu: Set XLEN independently from target
> > -------
> 
> Regards,
> Bin

-- 
Regards,
Atish

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Wed, Dec 23, 2020 at 3:59 AM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Tue, 2020-12-22 at 13:35 +0800, Bin Meng wrote:
> > Hi Atish,
> >
> > On Sat, Dec 19, 2020 at 3:46 AM Atish Patra <Atish.Patra@wdc.com>
> > wrote:
> > >
> > > On Fri, 2020-12-18 at 16:42 +0800, Bin Meng wrote:
> > > > Hi Atish,
> > > >
> > > > On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <Atish.Patra@wdc.com>
> > > > wrote:
> > > > >
> > > > > On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > > > > > Hi Atish,
> > > > > >
> > > > > > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <
> > > > > > Atish.Patra@wdc.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > > > > > Hi Atish,
> > > > > > > >
> > > > > > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra
> > > > > > > > <atish.patra@wdc.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Currently, we place the DTB at 2MB from 4GB or end of
> > > > > > > > > DRAM
> > > > > > > > > which
> > > > > > > > > ever is
> > > > > > > > > lesser. However, Linux kernel can address only 1GB of
> > > > > > > > > memory
> > > > > > > > > for
> > > > > > > > > RV32.
> > > > > > > > > Thus, it can not map anything beyond 3GB (assuming 2GB
> > > > > > > > > is
> > > > > > > > > the
> > > > > > > > > starting address).
> > > > > > > > > As a result, it can not process DT and panic if opensbi
> > > > > > > > > dynamic
> > > > > > > > > firmware
> > > > > > > > > is used.
> > > > > > > > >
> > > > > > > > > Fix this by placing the DTB at 2MB from 3GB or end of
> > > > > > > > > DRAM
> > > > > > > > > whichever is lower.
> > > > > > > > >
> > > > > > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > > > > > ---
> > > > > > > > >  hw/riscv/boot.c | 4 ++--
> > > > > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > > > >
> > > > > > > >
> > > > > > > > With this patch, 32-bit sifive_u still does not boot
> > > > > > > > kernel
> > > > > > > > with
> > > > > > > > the
> > > > > > > > following patch applied on 5.10:
> > > > > > > >
> > > > > > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > > > > > >
> > > > > > > > Command I used:
> > > > > > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5
> > > > > > > > -
> > > > > > > > kernel
> > > > > > > > arch/riscv/boot/Image
> > > > > > > >
> > > > > > > > 32-bit virt cannot boot the same kernel image with memory
> > > > > > > > set
> > > > > > > > to
> > > > > > > > 2G
> > > > > > > > either:
> > > > > > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -
> > > > > > > > kernel
> > > > > > > > arch/riscv/boot/Image
> > > > > > > >
> > > > > > >
> > > > > > > Hi Bin,
> > > > > > > As mentioned in the email on the linux mailing list, this
> > > > > > > patch
> > > > > > > only
> > > > > > > solves 2GB problem. sifive_u problem is solved by
> > > > > > > Alistair's
> > > > > > > patch[1].
> > > > > > >
> > > > > > > He is planning to send the PR soon. The issue with sifive_u
> > > > > > > boot
> > > > > > > was it
> > > > > > > was failing the 32 bit test earlier resulting a 2MB aligned
> > > > > > > address
> > > > > > > instead of 4MB.
> > > > > >
> > > > > > Ah, I see. However my testing shows that virt with 2G still
> > > > > > does
> > > > > > not
> > > > > > boot with this patch.
> > > > > >
> > > > >
> > > > > Strange. I verified again with following combination with -bios
> > > > > and
> > > > > without bios parameter.
> > > > >
> > > > > 1. virt 32/64 with 1GB/2GB memory
> > > > > 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's patch
> > > > > included)
> > > > >
> > > > > Can you share the boot log along with the head commit of Qemu
> > > > > and
> > > > > commandline ? I am using 5.10 kernel with my kernel fix.
> > > > >
> > > >
> > > > I was using Alistair's QEMU repo for testing and 5.10 kernel with
> > > > your
> > > > kernel fix:
> > > >
> > > > $ git checkout -b testing pull-riscv-to-apply-20201217-1
> > > > $ apply this patch
> > > > $ mkdir build;cd build;../configure
> > > > --target-list=riscv64-softmmu,riscv32-softmmu;make -j
> > > >
> > > > $ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > > > ~/work/git/linux/arch/riscv/boot/Image
> > > >
> > > > OpenSBI v0.8
> > > >    ____                    _____ ____ _____
> > > >   / __ \                  / ____|  _ \_   _|
> > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > >         | |
> > > >         |_|
> > > >
> > > > Platform Name       : riscv-virtio,qemu
> > > > Platform Features   : timer,mfdeleg
> > > > Platform HART Count : 4
> > > > Boot HART ID        : 3
> > > > Boot HART ISA       : rv32imafdcsu
> > > > BOOT HART Features  : pmp,scounteren,mcounteren,time
> > > > BOOT HART PMP Count : 16
> > > > Firmware Base       : 0x80000000
> > > > Firmware Size       : 104 KB
> > > > Runtime SBI Version : 0.2
> > > >
> > > > MIDELEG : 0x00000222
> > > > MEDELEG : 0x0000b109
> > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > <hangs here>
> > > >
> > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5 -
> > > > kernel
> > > > ~/work/git/linux/arch/riscv/boot/Image
> > > >
> > > > OpenSBI v0.8
> > > >    ____                    _____ ____ _____
> > > >   / __ \                  / ____|  _ \_   _|
> > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > >         | |
> > > >         |_|
> > > >
> > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > Platform Features   : timer,mfdeleg
> > > > Platform HART Count : 5
> > > > Boot HART ID        : 4
> > > > Boot HART ISA       : rv32imafdcsu
> > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > BOOT HART PMP Count : 16
> > > > Firmware Base       : 0x80000000
> > > > Firmware Size       : 112 KB
> > > > Runtime SBI Version : 0.2
> > > >
> > > > MIDELEG : 0x00000222
> > > > MEDELEG : 0x0000b109
> > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > <hangs here>
> > > >
> > > > The following is sifive_u with 1G:
> > > >
> > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -
> > > > kernel
> > > > ~/work/git/linux/arch/riscv/boot/Image
> > > >
> > > > OpenSBI v0.8
> > > >    ____                    _____ ____ _____
> > > >   / __ \                  / ____|  _ \_   _|
> > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > >         | |
> > > >         |_|
> > > >
> > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > Platform Features   : timer,mfdeleg
> > > > Platform HART Count : 5
> > > > Boot HART ID        : 3
> > > > Boot HART ISA       : rv32imafdcsu
> > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > BOOT HART PMP Count : 16
> > > > Firmware Base       : 0x80000000
> > > > Firmware Size       : 112 KB
> > > > Runtime SBI Version : 0.2
> > > >
> > > > MIDELEG : 0x00000222
> > > > MEDELEG : 0x0000b109
> > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > [    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
> > > > (bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld (GNU
> > > > Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
> > > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 -
> > > > 0x80400000
> > > > [    0.000000] efi: UEFI not found.
> > > > [    0.000000] Zone ranges:
> > > > [    0.000000]   Normal   [mem 0x0000000080400000-
> > > > 0x00000000bfffffff]
> > > > [    0.000000] Movable zone start for each node
> > > > [    0.000000] Early memory node ranges
> > > > [    0.000000]   node   0: [mem 0x0000000080400000-
> > > > 0x00000000bfffffff]
> > > > [    0.000000] Initmem setup node 0 [mem 0x0000000080400000-
> > > > 0x00000000bfffffff]
> > > > [    0.000000] SBI specification v0.2 detected
> > > > [    0.000000] SBI implementation ID=0x1 Version=0x8
> > > > [    0.000000] SBI v0.2 TIME extension detected
> > > > [    0.000000] SBI v0.2 IPI extension detected
> > > > [    0.000000] SBI v0.2 RFENCE extension detected
> > > > [    0.000000] SBI v0.2 HSM extension detected
> > > > [    0.000000] CPU with hartid=0 is not available
> > > > [    0.000000] CPU with hartid=0 is not available
> > > >
> > > > Regards,
> > > > Bin
> > >
> > > May be you forgot to apply this patch for 2GB case on top of
> > > Alistair's
> > > tree? I don't see any issues with exact same setup.
> >
> > That's really weird. I have:
> >
> > $ git log --oneline
> > 3ced2fb RISC-V: Place DTB at 3GB boundary instead of 4GB
> > d31e970 riscv/opentitan: Update the OpenTitan memory layout
> > 3ed2b8a hw/riscv: Use the CPU to determine if 32-bit
> > 094b072 target/riscv: cpu: Set XLEN independently from target
> > 8987cdc4 target/riscv: csr: Remove compile time XLEN checks
> >
> > I just rebuilt the QEMU binaries but still have the same result. 2G
> > does not boot on either 'virt' or 'sifive_u'.
> >
> > $ ./qemu-system-riscv32 -version
> > QEMU emulator version 5.2.50 (v5.2.0-551-g3ced2fb)
> > Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project
> > developers
> >
> > Maybe it's toolchain related? I am using kernel.org toolchain
> > riscv64-linux-gcc (GCC) 8.1.0. I see you were using
> > riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0.
> >
>
> Strange. If it is a toolchain related issue, the cause might be
> completely different. Can you give it a try to with updated toolchain ?
> The pre-built toolchain available in bootlin[1] is 10.2.0.
>
> [1] https://toolchains.bootlin.com/

I will give it a try.

>
> Can I download pre-built gcc 8.1.0 toolchain from somewhere ?

Yes, you can download it from
http://cdn.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.1.0/x86_64-gcc-8.1.0-nolibc-riscv64-linux.tar.xz

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Wed, Dec 23, 2020 at 9:20 AM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Atish,
>
> On Wed, Dec 23, 2020 at 3:59 AM Atish Patra <Atish.Patra@wdc.com> wrote:
> >
> > On Tue, 2020-12-22 at 13:35 +0800, Bin Meng wrote:
> > > Hi Atish,
> > >
> > > On Sat, Dec 19, 2020 at 3:46 AM Atish Patra <Atish.Patra@wdc.com>
> > > wrote:
> > > >
> > > > On Fri, 2020-12-18 at 16:42 +0800, Bin Meng wrote:
> > > > > Hi Atish,
> > > > >
> > > > > On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <Atish.Patra@wdc.com>
> > > > > wrote:
> > > > > >
> > > > > > On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > > > > > > Hi Atish,
> > > > > > >
> > > > > > > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <
> > > > > > > Atish.Patra@wdc.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > > > > > > Hi Atish,
> > > > > > > > >
> > > > > > > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra
> > > > > > > > > <atish.patra@wdc.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Currently, we place the DTB at 2MB from 4GB or end of
> > > > > > > > > > DRAM
> > > > > > > > > > which
> > > > > > > > > > ever is
> > > > > > > > > > lesser. However, Linux kernel can address only 1GB of
> > > > > > > > > > memory
> > > > > > > > > > for
> > > > > > > > > > RV32.
> > > > > > > > > > Thus, it can not map anything beyond 3GB (assuming 2GB
> > > > > > > > > > is
> > > > > > > > > > the
> > > > > > > > > > starting address).
> > > > > > > > > > As a result, it can not process DT and panic if opensbi
> > > > > > > > > > dynamic
> > > > > > > > > > firmware
> > > > > > > > > > is used.
> > > > > > > > > >
> > > > > > > > > > Fix this by placing the DTB at 2MB from 3GB or end of
> > > > > > > > > > DRAM
> > > > > > > > > > whichever is lower.
> > > > > > > > > >
> > > > > > > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > > > > > > ---
> > > > > > > > > >  hw/riscv/boot.c | 4 ++--
> > > > > > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > With this patch, 32-bit sifive_u still does not boot
> > > > > > > > > kernel
> > > > > > > > > with
> > > > > > > > > the
> > > > > > > > > following patch applied on 5.10:
> > > > > > > > >
> > > > > > > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > > > > > > >
> > > > > > > > > Command I used:
> > > > > > > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5
> > > > > > > > > -
> > > > > > > > > kernel
> > > > > > > > > arch/riscv/boot/Image
> > > > > > > > >
> > > > > > > > > 32-bit virt cannot boot the same kernel image with memory
> > > > > > > > > set
> > > > > > > > > to
> > > > > > > > > 2G
> > > > > > > > > either:
> > > > > > > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -
> > > > > > > > > kernel
> > > > > > > > > arch/riscv/boot/Image
> > > > > > > > >
> > > > > > > >
> > > > > > > > Hi Bin,
> > > > > > > > As mentioned in the email on the linux mailing list, this
> > > > > > > > patch
> > > > > > > > only
> > > > > > > > solves 2GB problem. sifive_u problem is solved by
> > > > > > > > Alistair's
> > > > > > > > patch[1].
> > > > > > > >
> > > > > > > > He is planning to send the PR soon. The issue with sifive_u
> > > > > > > > boot
> > > > > > > > was it
> > > > > > > > was failing the 32 bit test earlier resulting a 2MB aligned
> > > > > > > > address
> > > > > > > > instead of 4MB.
> > > > > > >
> > > > > > > Ah, I see. However my testing shows that virt with 2G still
> > > > > > > does
> > > > > > > not
> > > > > > > boot with this patch.
> > > > > > >
> > > > > >
> > > > > > Strange. I verified again with following combination with -bios
> > > > > > and
> > > > > > without bios parameter.
> > > > > >
> > > > > > 1. virt 32/64 with 1GB/2GB memory
> > > > > > 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's patch
> > > > > > included)
> > > > > >
> > > > > > Can you share the boot log along with the head commit of Qemu
> > > > > > and
> > > > > > commandline ? I am using 5.10 kernel with my kernel fix.
> > > > > >
> > > > >
> > > > > I was using Alistair's QEMU repo for testing and 5.10 kernel with
> > > > > your
> > > > > kernel fix:
> > > > >
> > > > > $ git checkout -b testing pull-riscv-to-apply-20201217-1
> > > > > $ apply this patch
> > > > > $ mkdir build;cd build;../configure
> > > > > --target-list=riscv64-softmmu,riscv32-softmmu;make -j
> > > > >
> > > > > $ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -kernel
> > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > >
> > > > > OpenSBI v0.8
> > > > >    ____                    _____ ____ _____
> > > > >   / __ \                  / ____|  _ \_   _|
> > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > >         | |
> > > > >         |_|
> > > > >
> > > > > Platform Name       : riscv-virtio,qemu
> > > > > Platform Features   : timer,mfdeleg
> > > > > Platform HART Count : 4
> > > > > Boot HART ID        : 3
> > > > > Boot HART ISA       : rv32imafdcsu
> > > > > BOOT HART Features  : pmp,scounteren,mcounteren,time
> > > > > BOOT HART PMP Count : 16
> > > > > Firmware Base       : 0x80000000
> > > > > Firmware Size       : 104 KB
> > > > > Runtime SBI Version : 0.2
> > > > >
> > > > > MIDELEG : 0x00000222
> > > > > MEDELEG : 0x0000b109
> > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > <hangs here>
> > > > >
> > > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5 -
> > > > > kernel
> > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > >
> > > > > OpenSBI v0.8
> > > > >    ____                    _____ ____ _____
> > > > >   / __ \                  / ____|  _ \_   _|
> > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > >         | |
> > > > >         |_|
> > > > >
> > > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > > Platform Features   : timer,mfdeleg
> > > > > Platform HART Count : 5
> > > > > Boot HART ID        : 4
> > > > > Boot HART ISA       : rv32imafdcsu
> > > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > > BOOT HART PMP Count : 16
> > > > > Firmware Base       : 0x80000000
> > > > > Firmware Size       : 112 KB
> > > > > Runtime SBI Version : 0.2
> > > > >
> > > > > MIDELEG : 0x00000222
> > > > > MEDELEG : 0x0000b109
> > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > <hangs here>
> > > > >
> > > > > The following is sifive_u with 1G:
> > > > >
> > > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5 -
> > > > > kernel
> > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > >
> > > > > OpenSBI v0.8
> > > > >    ____                    _____ ____ _____
> > > > >   / __ \                  / ____|  _ \_   _|
> > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > >         | |
> > > > >         |_|
> > > > >
> > > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > > Platform Features   : timer,mfdeleg
> > > > > Platform HART Count : 5
> > > > > Boot HART ID        : 3
> > > > > Boot HART ISA       : rv32imafdcsu
> > > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > > BOOT HART PMP Count : 16
> > > > > Firmware Base       : 0x80000000
> > > > > Firmware Size       : 112 KB
> > > > > Runtime SBI Version : 0.2
> > > > >
> > > > > MIDELEG : 0x00000222
> > > > > MEDELEG : 0x0000b109
> > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > [    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
> > > > > (bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld (GNU
> > > > > Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
> > > > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 -
> > > > > 0x80400000
> > > > > [    0.000000] efi: UEFI not found.
> > > > > [    0.000000] Zone ranges:
> > > > > [    0.000000]   Normal   [mem 0x0000000080400000-
> > > > > 0x00000000bfffffff]
> > > > > [    0.000000] Movable zone start for each node
> > > > > [    0.000000] Early memory node ranges
> > > > > [    0.000000]   node   0: [mem 0x0000000080400000-
> > > > > 0x00000000bfffffff]
> > > > > [    0.000000] Initmem setup node 0 [mem 0x0000000080400000-
> > > > > 0x00000000bfffffff]
> > > > > [    0.000000] SBI specification v0.2 detected
> > > > > [    0.000000] SBI implementation ID=0x1 Version=0x8
> > > > > [    0.000000] SBI v0.2 TIME extension detected
> > > > > [    0.000000] SBI v0.2 IPI extension detected
> > > > > [    0.000000] SBI v0.2 RFENCE extension detected
> > > > > [    0.000000] SBI v0.2 HSM extension detected
> > > > > [    0.000000] CPU with hartid=0 is not available
> > > > > [    0.000000] CPU with hartid=0 is not available
> > > > >
> > > > > Regards,
> > > > > Bin
> > > >
> > > > May be you forgot to apply this patch for 2GB case on top of
> > > > Alistair's
> > > > tree? I don't see any issues with exact same setup.
> > >
> > > That's really weird. I have:
> > >
> > > $ git log --oneline
> > > 3ced2fb RISC-V: Place DTB at 3GB boundary instead of 4GB
> > > d31e970 riscv/opentitan: Update the OpenTitan memory layout
> > > 3ed2b8a hw/riscv: Use the CPU to determine if 32-bit
> > > 094b072 target/riscv: cpu: Set XLEN independently from target
> > > 8987cdc4 target/riscv: csr: Remove compile time XLEN checks
> > >
> > > I just rebuilt the QEMU binaries but still have the same result. 2G
> > > does not boot on either 'virt' or 'sifive_u'.
> > >
> > > $ ./qemu-system-riscv32 -version
> > > QEMU emulator version 5.2.50 (v5.2.0-551-g3ced2fb)
> > > Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project
> > > developers
> > >
> > > Maybe it's toolchain related? I am using kernel.org toolchain
> > > riscv64-linux-gcc (GCC) 8.1.0. I see you were using
> > > riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0.
> > >
> >
> > Strange. If it is a toolchain related issue, the cause might be
> > completely different. Can you give it a try to with updated toolchain ?
> > The pre-built toolchain available in bootlin[1] is 10.2.0.
> >
> > [1] https://toolchains.bootlin.com/
>
> I will give it a try.

I used the bootlin pre-built 10.2.0 toolchain to build the Linux
kernel. Still, 2GB does not boot on QEMU.

I looked at your kernel log again, and found:

[    0.000000] Linux version 5.10.0-00022-ge20097fb37e2-dirty

It seems you have 22 commits on top of 5.10.

I only have one commit "RISC-V: Fix usage of
memblock_enforce_memory_limit" on top of 5.10.

Could you please check?

>
> >
> > Can I download pre-built gcc 8.1.0 toolchain from somewhere ?
>
> Yes, you can download it from
> http://cdn.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.1.0/x86_64-gcc-8.1.0-nolibc-riscv64-linux.tar.xz

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Tue, 2020-12-29 at 12:49 +0800, Bin Meng wrote:
> Hi Atish,
> 
> On Wed, Dec 23, 2020 at 9:20 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > 
> > Hi Atish,
> > 
> > On Wed, Dec 23, 2020 at 3:59 AM Atish Patra <Atish.Patra@wdc.com>
> > wrote:
> > > 
> > > On Tue, 2020-12-22 at 13:35 +0800, Bin Meng wrote:
> > > > Hi Atish,
> > > > 
> > > > On Sat, Dec 19, 2020 at 3:46 AM Atish Patra <
> > > > Atish.Patra@wdc.com>
> > > > wrote:
> > > > > 
> > > > > On Fri, 2020-12-18 at 16:42 +0800, Bin Meng wrote:
> > > > > > Hi Atish,
> > > > > > 
> > > > > > On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <
> > > > > > Atish.Patra@wdc.com>
> > > > > > wrote:
> > > > > > > 
> > > > > > > On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > > > > > > > Hi Atish,
> > > > > > > > 
> > > > > > > > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <
> > > > > > > > Atish.Patra@wdc.com>
> > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > > > > > > > Hi Atish,
> > > > > > > > > > 
> > > > > > > > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra
> > > > > > > > > > <atish.patra@wdc.com>
> > > > > > > > > > wrote:
> > > > > > > > > > > 
> > > > > > > > > > > Currently, we place the DTB at 2MB from 4GB or
> > > > > > > > > > > end of
> > > > > > > > > > > DRAM
> > > > > > > > > > > which
> > > > > > > > > > > ever is
> > > > > > > > > > > lesser. However, Linux kernel can address only
> > > > > > > > > > > 1GB of
> > > > > > > > > > > memory
> > > > > > > > > > > for
> > > > > > > > > > > RV32.
> > > > > > > > > > > Thus, it can not map anything beyond 3GB
> > > > > > > > > > > (assuming 2GB
> > > > > > > > > > > is
> > > > > > > > > > > the
> > > > > > > > > > > starting address).
> > > > > > > > > > > As a result, it can not process DT and panic if
> > > > > > > > > > > opensbi
> > > > > > > > > > > dynamic
> > > > > > > > > > > firmware
> > > > > > > > > > > is used.
> > > > > > > > > > > 
> > > > > > > > > > > Fix this by placing the DTB at 2MB from 3GB or
> > > > > > > > > > > end of
> > > > > > > > > > > DRAM
> > > > > > > > > > > whichever is lower.
> > > > > > > > > > > 
> > > > > > > > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > > > > > > > ---
> > > > > > > > > > >  hw/riscv/boot.c | 4 ++--
> > > > > > > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > With this patch, 32-bit sifive_u still does not
> > > > > > > > > > boot
> > > > > > > > > > kernel
> > > > > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > following patch applied on 5.10:
> > > > > > > > > > 
> > > > > > > > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > > > > > > > > 
> > > > > > > > > > Command I used:
> > > > > > > > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G
> > > > > > > > > > -smp 5
> > > > > > > > > > -
> > > > > > > > > > kernel
> > > > > > > > > > arch/riscv/boot/Image
> > > > > > > > > > 
> > > > > > > > > > 32-bit virt cannot boot the same kernel image with
> > > > > > > > > > memory
> > > > > > > > > > set
> > > > > > > > > > to
> > > > > > > > > > 2G
> > > > > > > > > > either:
> > > > > > > > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp
> > > > > > > > > > 4 -
> > > > > > > > > > kernel
> > > > > > > > > > arch/riscv/boot/Image
> > > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Hi Bin,
> > > > > > > > > As mentioned in the email on the linux mailing list,
> > > > > > > > > this
> > > > > > > > > patch
> > > > > > > > > only
> > > > > > > > > solves 2GB problem. sifive_u problem is solved by
> > > > > > > > > Alistair's
> > > > > > > > > patch[1].
> > > > > > > > > 
> > > > > > > > > He is planning to send the PR soon. The issue with
> > > > > > > > > sifive_u
> > > > > > > > > boot
> > > > > > > > > was it
> > > > > > > > > was failing the 32 bit test earlier resulting a 2MB
> > > > > > > > > aligned
> > > > > > > > > address
> > > > > > > > > instead of 4MB.
> > > > > > > > 
> > > > > > > > Ah, I see. However my testing shows that virt with 2G
> > > > > > > > still
> > > > > > > > does
> > > > > > > > not
> > > > > > > > boot with this patch.
> > > > > > > > 
> > > > > > > 
> > > > > > > Strange. I verified again with following combination with
> > > > > > > -bios
> > > > > > > and
> > > > > > > without bios parameter.
> > > > > > > 
> > > > > > > 1. virt 32/64 with 1GB/2GB memory
> > > > > > > 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's
> > > > > > > patch
> > > > > > > included)
> > > > > > > 
> > > > > > > Can you share the boot log along with the head commit of
> > > > > > > Qemu
> > > > > > > and
> > > > > > > commandline ? I am using 5.10 kernel with my kernel fix.
> > > > > > > 
> > > > > > 
> > > > > > I was using Alistair's QEMU repo for testing and 5.10
> > > > > > kernel with
> > > > > > your
> > > > > > kernel fix:
> > > > > > 
> > > > > > $ git checkout -b testing pull-riscv-to-apply-20201217-1
> > > > > > $ apply this patch
> > > > > > $ mkdir build;cd build;../configure
> > > > > > --target-list=riscv64-softmmu,riscv32-softmmu;make -j
> > > > > > 
> > > > > > $ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -
> > > > > > kernel
> > > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > > > 
> > > > > > OpenSBI v0.8
> > > > > >    ____                    _____ ____ _____
> > > > > >   / __ \                  / ____|  _ \_   _|
> > > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > > >         | |
> > > > > >         |_|
> > > > > > 
> > > > > > Platform Name       : riscv-virtio,qemu
> > > > > > Platform Features   : timer,mfdeleg
> > > > > > Platform HART Count : 4
> > > > > > Boot HART ID        : 3
> > > > > > Boot HART ISA       : rv32imafdcsu
> > > > > > BOOT HART Features  : pmp,scounteren,mcounteren,time
> > > > > > BOOT HART PMP Count : 16
> > > > > > Firmware Base       : 0x80000000
> > > > > > Firmware Size       : 104 KB
> > > > > > Runtime SBI Version : 0.2
> > > > > > 
> > > > > > MIDELEG : 0x00000222
> > > > > > MEDELEG : 0x0000b109
> > > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > > <hangs here>
> > > > > > 
> > > > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5
> > > > > > -
> > > > > > kernel
> > > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > > > 
> > > > > > OpenSBI v0.8
> > > > > >    ____                    _____ ____ _____
> > > > > >   / __ \                  / ____|  _ \_   _|
> > > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > > >         | |
> > > > > >         |_|
> > > > > > 
> > > > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > > > Platform Features   : timer,mfdeleg
> > > > > > Platform HART Count : 5
> > > > > > Boot HART ID        : 4
> > > > > > Boot HART ISA       : rv32imafdcsu
> > > > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > > > BOOT HART PMP Count : 16
> > > > > > Firmware Base       : 0x80000000
> > > > > > Firmware Size       : 112 KB
> > > > > > Runtime SBI Version : 0.2
> > > > > > 
> > > > > > MIDELEG : 0x00000222
> > > > > > MEDELEG : 0x0000b109
> > > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > > <hangs here>
> > > > > > 
> > > > > > The following is sifive_u with 1G:
> > > > > > 
> > > > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5
> > > > > > -
> > > > > > kernel
> > > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > > > 
> > > > > > OpenSBI v0.8
> > > > > >    ____                    _____ ____ _____
> > > > > >   / __ \                  / ____|  _ \_   _|
> > > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > > >         | |
> > > > > >         |_|
> > > > > > 
> > > > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > > > Platform Features   : timer,mfdeleg
> > > > > > Platform HART Count : 5
> > > > > > Boot HART ID        : 3
> > > > > > Boot HART ISA       : rv32imafdcsu
> > > > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > > > BOOT HART PMP Count : 16
> > > > > > Firmware Base       : 0x80000000
> > > > > > Firmware Size       : 112 KB
> > > > > > Runtime SBI Version : 0.2
> > > > > > 
> > > > > > MIDELEG : 0x00000222
> > > > > > MEDELEG : 0x0000b109
> > > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > > [    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
> > > > > > (bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld
> > > > > > (GNU
> > > > > > Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
> > > > > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 -
> > > > > > 0x80400000
> > > > > > [    0.000000] efi: UEFI not found.
> > > > > > [    0.000000] Zone ranges:
> > > > > > [    0.000000]   Normal   [mem 0x0000000080400000-
> > > > > > 0x00000000bfffffff]
> > > > > > [    0.000000] Movable zone start for each node
> > > > > > [    0.000000] Early memory node ranges
> > > > > > [    0.000000]   node   0: [mem 0x0000000080400000-
> > > > > > 0x00000000bfffffff]
> > > > > > [    0.000000] Initmem setup node 0 [mem
> > > > > > 0x0000000080400000-
> > > > > > 0x00000000bfffffff]
> > > > > > [    0.000000] SBI specification v0.2 detected
> > > > > > [    0.000000] SBI implementation ID=0x1 Version=0x8
> > > > > > [    0.000000] SBI v0.2 TIME extension detected
> > > > > > [    0.000000] SBI v0.2 IPI extension detected
> > > > > > [    0.000000] SBI v0.2 RFENCE extension detected
> > > > > > [    0.000000] SBI v0.2 HSM extension detected
> > > > > > [    0.000000] CPU with hartid=0 is not available
> > > > > > [    0.000000] CPU with hartid=0 is not available
> > > > > > 
> > > > > > Regards,
> > > > > > Bin
> > > > > 
> > > > > May be you forgot to apply this patch for 2GB case on top of
> > > > > Alistair's
> > > > > tree? I don't see any issues with exact same setup.
> > > > 
> > > > That's really weird. I have:
> > > > 
> > > > $ git log --oneline
> > > > 3ced2fb RISC-V: Place DTB at 3GB boundary instead of 4GB
> > > > d31e970 riscv/opentitan: Update the OpenTitan memory layout
> > > > 3ed2b8a hw/riscv: Use the CPU to determine if 32-bit
> > > > 094b072 target/riscv: cpu: Set XLEN independently from target
> > > > 8987cdc4 target/riscv: csr: Remove compile time XLEN checks
> > > > 
> > > > I just rebuilt the QEMU binaries but still have the same
> > > > result. 2G
> > > > does not boot on either 'virt' or 'sifive_u'.
> > > > 
> > > > $ ./qemu-system-riscv32 -version
> > > > QEMU emulator version 5.2.50 (v5.2.0-551-g3ced2fb)
> > > > Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project
> > > > developers
> > > > 
> > > > Maybe it's toolchain related? I am using kernel.org toolchain
> > > > riscv64-linux-gcc (GCC) 8.1.0. I see you were using
> > > > riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0.
> > > > 
> > > 
> > > Strange. If it is a toolchain related issue, the cause might be
> > > completely different. Can you give it a try to with updated
> > > toolchain ?
> > > The pre-built toolchain available in bootlin[1] is 10.2.0.
> > > 
> > > [1] https://toolchains.bootlin.com/
> > 
> > I will give it a try.
> 
> I used the bootlin pre-built 10.2.0 toolchain to build the Linux
> kernel. Still, 2GB does not boot on QEMU.
> 
> I looked at your kernel log again, and found:
> 
> [    0.000000] Linux version 5.10.0-00022-ge20097fb37e2-dirty
> 
> It seems you have 22 commits on top of 5.10.
> 
> I only have one commit "RISC-V: Fix usage of
> memblock_enforce_memory_limit" on top of 5.10.
> 
> Could you please check?
> 

Sorry for the late reply. I was on vacation. I was using for-next
branch on top of 5.10. I just tested with 5.11-rc2 and everything seems
to be fine. Can you check with 5.11-rc2 ?

Apologies for the any confusion caused.

> > 
> > > 
> > > Can I download pre-built gcc 8.1.0 toolchain from somewhere ?
> > 
> > Yes, you can download it from
> > http://cdn.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.1.0/x86_64-gcc-8.1.0-nolibc-riscv64-linux.tar.xz
> 
> Regards,
> Bin

-- 
Regards,
Atish

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Tue, Jan 5, 2021 at 4:24 AM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Tue, 2020-12-29 at 12:49 +0800, Bin Meng wrote:
> > Hi Atish,
> >
> > On Wed, Dec 23, 2020 at 9:20 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > Hi Atish,
> > >
> > > On Wed, Dec 23, 2020 at 3:59 AM Atish Patra <Atish.Patra@wdc.com>
> > > wrote:
> > > >
> > > > On Tue, 2020-12-22 at 13:35 +0800, Bin Meng wrote:
> > > > > Hi Atish,
> > > > >
> > > > > On Sat, Dec 19, 2020 at 3:46 AM Atish Patra <
> > > > > Atish.Patra@wdc.com>
> > > > > wrote:
> > > > > >
> > > > > > On Fri, 2020-12-18 at 16:42 +0800, Bin Meng wrote:
> > > > > > > Hi Atish,
> > > > > > >
> > > > > > > On Fri, Dec 18, 2020 at 4:00 PM Atish Patra <
> > > > > > > Atish.Patra@wdc.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > On Fri, 2020-12-18 at 15:33 +0800, Bin Meng wrote:
> > > > > > > > > Hi Atish,
> > > > > > > > >
> > > > > > > > > On Fri, Dec 18, 2020 at 3:27 PM Atish Patra <
> > > > > > > > > Atish.Patra@wdc.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > On Fri, 2020-12-18 at 15:21 +0800, Bin Meng wrote:
> > > > > > > > > > > Hi Atish,
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra
> > > > > > > > > > > <atish.patra@wdc.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Currently, we place the DTB at 2MB from 4GB or
> > > > > > > > > > > > end of
> > > > > > > > > > > > DRAM
> > > > > > > > > > > > which
> > > > > > > > > > > > ever is
> > > > > > > > > > > > lesser. However, Linux kernel can address only
> > > > > > > > > > > > 1GB of
> > > > > > > > > > > > memory
> > > > > > > > > > > > for
> > > > > > > > > > > > RV32.
> > > > > > > > > > > > Thus, it can not map anything beyond 3GB
> > > > > > > > > > > > (assuming 2GB
> > > > > > > > > > > > is
> > > > > > > > > > > > the
> > > > > > > > > > > > starting address).
> > > > > > > > > > > > As a result, it can not process DT and panic if
> > > > > > > > > > > > opensbi
> > > > > > > > > > > > dynamic
> > > > > > > > > > > > firmware
> > > > > > > > > > > > is used.
> > > > > > > > > > > >
> > > > > > > > > > > > Fix this by placing the DTB at 2MB from 3GB or
> > > > > > > > > > > > end of
> > > > > > > > > > > > DRAM
> > > > > > > > > > > > whichever is lower.
> > > > > > > > > > > >
> > > > > > > > > > > > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > > > > > > > > > > > ---
> > > > > > > > > > > >  hw/riscv/boot.c | 4 ++--
> > > > > > > > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > With this patch, 32-bit sifive_u still does not
> > > > > > > > > > > boot
> > > > > > > > > > > kernel
> > > > > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > following patch applied on 5.10:
> > > > > > > > > > >
> > > > > > > > > > > https://patchwork.kernel.org/project/linux-riscv/patch/20201217074855.1948743-1-atish.patra@wdc.com/
> > > > > > > > > > >
> > > > > > > > > > > Command I used:
> > > > > > > > > > > $ qemu-system-riscv32 -nographic -M sifive_u -m 1G
> > > > > > > > > > > -smp 5
> > > > > > > > > > > -
> > > > > > > > > > > kernel
> > > > > > > > > > > arch/riscv/boot/Image
> > > > > > > > > > >
> > > > > > > > > > > 32-bit virt cannot boot the same kernel image with
> > > > > > > > > > > memory
> > > > > > > > > > > set
> > > > > > > > > > > to
> > > > > > > > > > > 2G
> > > > > > > > > > > either:
> > > > > > > > > > > $ qemu-system-riscv32 -nographic -M virt -m 2G -smp
> > > > > > > > > > > 4 -
> > > > > > > > > > > kernel
> > > > > > > > > > > arch/riscv/boot/Image
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi Bin,
> > > > > > > > > > As mentioned in the email on the linux mailing list,
> > > > > > > > > > this
> > > > > > > > > > patch
> > > > > > > > > > only
> > > > > > > > > > solves 2GB problem. sifive_u problem is solved by
> > > > > > > > > > Alistair's
> > > > > > > > > > patch[1].
> > > > > > > > > >
> > > > > > > > > > He is planning to send the PR soon. The issue with
> > > > > > > > > > sifive_u
> > > > > > > > > > boot
> > > > > > > > > > was it
> > > > > > > > > > was failing the 32 bit test earlier resulting a 2MB
> > > > > > > > > > aligned
> > > > > > > > > > address
> > > > > > > > > > instead of 4MB.
> > > > > > > > >
> > > > > > > > > Ah, I see. However my testing shows that virt with 2G
> > > > > > > > > still
> > > > > > > > > does
> > > > > > > > > not
> > > > > > > > > boot with this patch.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Strange. I verified again with following combination with
> > > > > > > > -bios
> > > > > > > > and
> > > > > > > > without bios parameter.
> > > > > > > >
> > > > > > > > 1. virt 32/64 with 1GB/2GB memory
> > > > > > > > 2. sifive_u 32/64 bit with 1GB/2GB memory (Alistair's
> > > > > > > > patch
> > > > > > > > included)
> > > > > > > >
> > > > > > > > Can you share the boot log along with the head commit of
> > > > > > > > Qemu
> > > > > > > > and
> > > > > > > > commandline ? I am using 5.10 kernel with my kernel fix.
> > > > > > > >
> > > > > > >
> > > > > > > I was using Alistair's QEMU repo for testing and 5.10
> > > > > > > kernel with
> > > > > > > your
> > > > > > > kernel fix:
> > > > > > >
> > > > > > > $ git checkout -b testing pull-riscv-to-apply-20201217-1
> > > > > > > $ apply this patch
> > > > > > > $ mkdir build;cd build;../configure
> > > > > > > --target-list=riscv64-softmmu,riscv32-softmmu;make -j
> > > > > > >
> > > > > > > $ ./qemu-system-riscv32 -nographic -M virt -m 2G -smp 4 -
> > > > > > > kernel
> > > > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > > > >
> > > > > > > OpenSBI v0.8
> > > > > > >    ____                    _____ ____ _____
> > > > > > >   / __ \                  / ____|  _ \_   _|
> > > > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > > > >         | |
> > > > > > >         |_|
> > > > > > >
> > > > > > > Platform Name       : riscv-virtio,qemu
> > > > > > > Platform Features   : timer,mfdeleg
> > > > > > > Platform HART Count : 4
> > > > > > > Boot HART ID        : 3
> > > > > > > Boot HART ISA       : rv32imafdcsu
> > > > > > > BOOT HART Features  : pmp,scounteren,mcounteren,time
> > > > > > > BOOT HART PMP Count : 16
> > > > > > > Firmware Base       : 0x80000000
> > > > > > > Firmware Size       : 104 KB
> > > > > > > Runtime SBI Version : 0.2
> > > > > > >
> > > > > > > MIDELEG : 0x00000222
> > > > > > > MEDELEG : 0x0000b109
> > > > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > > > <hangs here>
> > > > > > >
> > > > > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 2G -smp 5
> > > > > > > -
> > > > > > > kernel
> > > > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > > > >
> > > > > > > OpenSBI v0.8
> > > > > > >    ____                    _____ ____ _____
> > > > > > >   / __ \                  / ____|  _ \_   _|
> > > > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > > > >         | |
> > > > > > >         |_|
> > > > > > >
> > > > > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > > > > Platform Features   : timer,mfdeleg
> > > > > > > Platform HART Count : 5
> > > > > > > Boot HART ID        : 4
> > > > > > > Boot HART ISA       : rv32imafdcsu
> > > > > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > > > > BOOT HART PMP Count : 16
> > > > > > > Firmware Base       : 0x80000000
> > > > > > > Firmware Size       : 112 KB
> > > > > > > Runtime SBI Version : 0.2
> > > > > > >
> > > > > > > MIDELEG : 0x00000222
> > > > > > > MEDELEG : 0x0000b109
> > > > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > > > <hangs here>
> > > > > > >
> > > > > > > The following is sifive_u with 1G:
> > > > > > >
> > > > > > > $ ./qemu-system-riscv32 -nographic -M sifive_u -m 1G -smp 5
> > > > > > > -
> > > > > > > kernel
> > > > > > > ~/work/git/linux/arch/riscv/boot/Image
> > > > > > >
> > > > > > > OpenSBI v0.8
> > > > > > >    ____                    _____ ____ _____
> > > > > > >   / __ \                  / ____|  _ \_   _|
> > > > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > > > >         | |
> > > > > > >         |_|
> > > > > > >
> > > > > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > > > > Platform Features   : timer,mfdeleg
> > > > > > > Platform HART Count : 5
> > > > > > > Boot HART ID        : 3
> > > > > > > Boot HART ISA       : rv32imafdcsu
> > > > > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > > > > BOOT HART PMP Count : 16
> > > > > > > Firmware Base       : 0x80000000
> > > > > > > Firmware Size       : 112 KB
> > > > > > > Runtime SBI Version : 0.2
> > > > > > >
> > > > > > > MIDELEG : 0x00000222
> > > > > > > MEDELEG : 0x0000b109
> > > > > > > PMP0    : 0x80000000-0x8001ffff (A)
> > > > > > > PMP1    : 0x00000000-0xffffffff (A,R,W,X)
> > > > > > > [    0.000000] Linux version 5.10.0-00001-gbf0dad61896d
> > > > > > > (bmeng@pek-vx-bsp2) (riscv64-linux-gcc (GCC) 8.1.0, GNU ld
> > > > > > > (GNU
> > > > > > > Binutils) 2.30) #1 SMP Thu Dec 17 16:48:13 CST 2020
> > > > > > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 -
> > > > > > > 0x80400000
> > > > > > > [    0.000000] efi: UEFI not found.
> > > > > > > [    0.000000] Zone ranges:
> > > > > > > [    0.000000]   Normal   [mem 0x0000000080400000-
> > > > > > > 0x00000000bfffffff]
> > > > > > > [    0.000000] Movable zone start for each node
> > > > > > > [    0.000000] Early memory node ranges
> > > > > > > [    0.000000]   node   0: [mem 0x0000000080400000-
> > > > > > > 0x00000000bfffffff]
> > > > > > > [    0.000000] Initmem setup node 0 [mem
> > > > > > > 0x0000000080400000-
> > > > > > > 0x00000000bfffffff]
> > > > > > > [    0.000000] SBI specification v0.2 detected
> > > > > > > [    0.000000] SBI implementation ID=0x1 Version=0x8
> > > > > > > [    0.000000] SBI v0.2 TIME extension detected
> > > > > > > [    0.000000] SBI v0.2 IPI extension detected
> > > > > > > [    0.000000] SBI v0.2 RFENCE extension detected
> > > > > > > [    0.000000] SBI v0.2 HSM extension detected
> > > > > > > [    0.000000] CPU with hartid=0 is not available
> > > > > > > [    0.000000] CPU with hartid=0 is not available
> > > > > > >
> > > > > > > Regards,
> > > > > > > Bin
> > > > > >
> > > > > > May be you forgot to apply this patch for 2GB case on top of
> > > > > > Alistair's
> > > > > > tree? I don't see any issues with exact same setup.
> > > > >
> > > > > That's really weird. I have:
> > > > >
> > > > > $ git log --oneline
> > > > > 3ced2fb RISC-V: Place DTB at 3GB boundary instead of 4GB
> > > > > d31e970 riscv/opentitan: Update the OpenTitan memory layout
> > > > > 3ed2b8a hw/riscv: Use the CPU to determine if 32-bit
> > > > > 094b072 target/riscv: cpu: Set XLEN independently from target
> > > > > 8987cdc4 target/riscv: csr: Remove compile time XLEN checks
> > > > >
> > > > > I just rebuilt the QEMU binaries but still have the same
> > > > > result. 2G
> > > > > does not boot on either 'virt' or 'sifive_u'.
> > > > >
> > > > > $ ./qemu-system-riscv32 -version
> > > > > QEMU emulator version 5.2.50 (v5.2.0-551-g3ced2fb)
> > > > > Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project
> > > > > developers
> > > > >
> > > > > Maybe it's toolchain related? I am using kernel.org toolchain
> > > > > riscv64-linux-gcc (GCC) 8.1.0. I see you were using
> > > > > riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0.
> > > > >
> > > >
> > > > Strange. If it is a toolchain related issue, the cause might be
> > > > completely different. Can you give it a try to with updated
> > > > toolchain ?
> > > > The pre-built toolchain available in bootlin[1] is 10.2.0.
> > > >
> > > > [1] https://toolchains.bootlin.com/
> > >
> > > I will give it a try.
> >
> > I used the bootlin pre-built 10.2.0 toolchain to build the Linux
> > kernel. Still, 2GB does not boot on QEMU.
> >
> > I looked at your kernel log again, and found:
> >
> > [    0.000000] Linux version 5.10.0-00022-ge20097fb37e2-dirty
> >
> > It seems you have 22 commits on top of 5.10.
> >
> > I only have one commit "RISC-V: Fix usage of
> > memblock_enforce_memory_limit" on top of 5.10.
> >
> > Could you please check?
> >
>
> Sorry for the late reply. I was on vacation. I was using for-next
> branch on top of 5.10. I just tested with 5.11-rc2 and everything seems
> to be fine. Can you check with 5.11-rc2 ?

I just tested v5.11-rc2 and "-m 2G" boots now.

> Apologies for the any confusion caused.
>

So it looks there are multiple kernel breakages on 32-bit support.

Could you identify which kernel commit "fixed" things in your "Linux
version 5.10.0-00022-ge20097fb37e2-dirty" build since there are only
22 commits on top of 5.10?

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com> wrote:
>
> Currently, we place the DTB at 2MB from 4GB or end of DRAM which ever is
> lesser. However, Linux kernel can address only 1GB of memory for RV32.
> Thus, it can not map anything beyond 3GB (assuming 2GB is the starting address).
> As a result, it can not process DT and panic if opensbi dynamic firmware
> is used.
>
> Fix this by placing the DTB at 2MB from 3GB or end of DRAM whichever is lower.
>

Fixes: 66b1205bc5ab ("RISC-V: Copy the fdt in dram instead of ROM")

> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
>  hw/riscv/boot.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>

Reviewed-by: Bin Meng <bin.meng@windriver.com>
Tested-by: Bin Meng <bin.meng@windriver.com>

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Alistair Francis]

On Mon, Jan 4, 2021 at 7:11 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com> wrote:
> >
> > Currently, we place the DTB at 2MB from 4GB or end of DRAM which ever is
> > lesser. However, Linux kernel can address only 1GB of memory for RV32.
> > Thus, it can not map anything beyond 3GB (assuming 2GB is the starting address).
> > As a result, it can not process DT and panic if opensbi dynamic firmware
> > is used.
> >
> > Fix this by placing the DTB at 2MB from 3GB or end of DRAM whichever is lower.
> >
>
> Fixes: 66b1205bc5ab ("RISC-V: Copy the fdt in dram instead of ROM")
>
> > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > ---
> >  hw/riscv/boot.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
>
> Reviewed-by: Bin Meng <bin.meng@windriver.com>
> Tested-by: Bin Meng <bin.meng@windriver.com>

Thanks!

Applied to riscv-to-apply.next

Alistair

>

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Tue, 2021-01-05 at 11:11 +0800, Bin Meng wrote:
> On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com>
> wrote:
> > 
> > Currently, we place the DTB at 2MB from 4GB or end of DRAM which
> > ever is
> > lesser. However, Linux kernel can address only 1GB of memory for
> > RV32.
> > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > starting address).
> > As a result, it can not process DT and panic if opensbi dynamic
> > firmware
> > is used.
> > 
> > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > whichever is lower.
> > 
> 
> Fixes: 66b1205bc5ab ("RISC-V: Copy the fdt in dram instead of ROM")
> 

Just to clarify, it is not a bug in the original patch. This bug
appeared in recent kernel because kernel(from v5.10) doesn't use fixmap
for DT processing anymore. Thus, the DT has to be placed at location
that can be addressed by kernel. 

I will add the fixes tag.

> > Signed-off-by: Atish Patra <atish.patra@wdc.com>
> > ---
> >  hw/riscv/boot.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> 
> Reviewed-by: Bin Meng <bin.meng@windriver.com>
> Tested-by: Bin Meng <bin.meng@windriver.com>

Thanks.

-- 
Regards,
Atish

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Bin Meng]

Hi Atish,

On Wed, Jan 6, 2021 at 7:44 AM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Tue, 2021-01-05 at 11:11 +0800, Bin Meng wrote:
> > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com>
> > wrote:
> > >
> > > Currently, we place the DTB at 2MB from 4GB or end of DRAM which
> > > ever is
> > > lesser. However, Linux kernel can address only 1GB of memory for
> > > RV32.
> > > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > > starting address).
> > > As a result, it can not process DT and panic if opensbi dynamic
> > > firmware
> > > is used.
> > >
> > > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > > whichever is lower.
> > >
> >
> > Fixes: 66b1205bc5ab ("RISC-V: Copy the fdt in dram instead of ROM")
> >
>
> Just to clarify, it is not a bug in the original patch. This bug
> appeared in recent kernel because kernel(from v5.10) doesn't use fixmap
> for DT processing anymore. Thus, the DT has to be placed at location
> that can be addressed by kernel.

Thanks for the clarification.

>
> I will add the fixes tag.
>

Did you get a chance to trace to another possible kernel bug?

Regards,
Bin

Re: [PATCH] RISC-V: Place DTB at 3GB boundary instead of 4GB

[Atish Patra]

On Wed, 2021-01-06 at 08:04 +0800, Bin Meng wrote:
> Hi Atish,
> 
> On Wed, Jan 6, 2021 at 7:44 AM Atish Patra <Atish.Patra@wdc.com>
> wrote:
> > 
> > On Tue, 2021-01-05 at 11:11 +0800, Bin Meng wrote:
> > > On Fri, Dec 18, 2020 at 5:48 AM Atish Patra <atish.patra@wdc.com>
> > > wrote:
> > > > 
> > > > Currently, we place the DTB at 2MB from 4GB or end of DRAM
> > > > which
> > > > ever is
> > > > lesser. However, Linux kernel can address only 1GB of memory
> > > > for
> > > > RV32.
> > > > Thus, it can not map anything beyond 3GB (assuming 2GB is the
> > > > starting address).
> > > > As a result, it can not process DT and panic if opensbi dynamic
> > > > firmware
> > > > is used.
> > > > 
> > > > Fix this by placing the DTB at 2MB from 3GB or end of DRAM
> > > > whichever is lower.
> > > > 
> > > 
> > > Fixes: 66b1205bc5ab ("RISC-V: Copy the fdt in dram instead of
> > > ROM")
> > > 
> > 
> > Just to clarify, it is not a bug in the original patch. This bug
> > appeared in recent kernel because kernel(from v5.10) doesn't use
> > fixmap
> > for DT processing anymore. Thus, the DT has to be placed at
> > location
> > that can be addressed by kernel.
> 
> Thanks for the clarification.
> 
> > 
> > I will add the fixes tag.
> > 
> 
> Did you get a chance to trace to another possible kernel bug?
> 

Yeah. Let me explain. Apologies for long explanation in advance :)

5.10 kernel + memblock_enforce_memory_limit patch actually boots but
crashes after a point with 2GB of memory. You did not see any boot logs
as you were not using earlycon argument.

The issue was masked by the a kernel patch [3] that got merged in 5.11-
rc1. The patch[3] also throws a warning on RV32 because of a another
bug[1].

The real issue with 2GB memory in RV32 is that Linux kernel treats any
pointers within last 4K byte of addressable memory as error [2].

In the absence of the patch[3], kernel unflattens the DT towards the
end of the memory. These memory regions are absolutely fine and
addressable. I even verified the memory region in qemu and can print
the DT strings using gdb. However, kernel thinks as error by mistake
because of [2].

The kernel patch[3] masked off the 2GB problem because it tried to
allocate bunch of memblocks and failed. That lead memblock to allocate
memory for unflattend DT at a different place which helped. 

I have a fix solve the bug introduced by that patch[3] and can
reproduce the 2GB problem with 5.11-rc2 as well with the kernel fix. 

The simple solution is here to place the DTB at an address farther away
from the end of 3GB. Kernel puts some other data structure towards the
end of the addressable memory but never accesses those last 4K bytes of
memory. Thus we don't see a problem anymore. This problem manifests
only when you have a virtual address within 4K of addressable memory.
It is only possible with RV32 and 2GB combination.

fdt_addr = QEMU_ALIGN_DOWN(temp - fdtsize, X * MiB); 

where X must be > 4.

I tested with 8/16/32 MB aligned address and it works fine every time.
I think the safest option should be 32MB offset.

Kernel should also be fixed so that memblock allocation code should not
allocate anything within last 4K of addressable memory.

[1]
http://lists.infradead.org/pipermail/linux-riscv/2021-January/004086.html
[2]
https://github.com/torvalds/linux/blob/master/include/linux/err.h#L22

[3] 0f522592ab9f RISC-V: Add kernel image sections to the resource tree

> Regards,
> Bin

-- 
Regards,
Atish