[PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Anup Patel]

Currently, the range and default value of NR_CPUS is too restrictive
for high-end RISC-V systems with large number of HARTs. The latest
QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
restrictive for QEMU as well. Other major architectures (such as
ARM64, x86_64, MIPS, etc) have a much higher range and default
value of NR_CPUS.

This patch increases NR_CPUS range to 2-512 and default value to
XLEN (i.e. 32 for RV32 and 64 for RV64).

Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
Changes since v2:
 - Rebased on Linux-5.18-rc3
 - Use a different range when SBI v0.1 is enabled
Changes since v1:
 - Updated NR_CPUS range to 2-512 which reflects maximum number of
   CPUs supported by QEMU virt machine.
---
 arch/riscv/Kconfig | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 00fd9c548f26..1823f281069f 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -275,10 +275,13 @@ config SMP
 	  If you don't know what to do here, say N.
 
 config NR_CPUS
-	int "Maximum number of CPUs (2-32)"
-	range 2 32
+	int "Maximum number of CPUs (2-512)"
 	depends on SMP
-	default "8"
+	range 2 512 if !SBI_V01
+	range 2 32 if SBI_V01 && 32BIT
+	range 2 64 if SBI_V01 && 64BIT
+	default "32" if 32BIT
+	default "64" if 64BIT
 
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Heinrich Schuchardt]

On 4/20/22 13:24, Anup Patel wrote:
> Currently, the range and default value of NR_CPUS is too restrictive
> for high-end RISC-V systems with large number of HARTs. The latest
> QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> restrictive for QEMU as well. Other major architectures (such as
> ARM64, x86_64, MIPS, etc) have a much higher range and default
> value of NR_CPUS.
> 
> This patch increases NR_CPUS range to 2-512 and default value to
> XLEN (i.e. 32 for RV32 and 64 for RV64).
> 
> Signed-off-by: Anup Patel <apatel@ventanamicro.com>

Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>

> ---
> Changes since v2:
>   - Rebased on Linux-5.18-rc3
>   - Use a different range when SBI v0.1 is enabled
> Changes since v1:
>   - Updated NR_CPUS range to 2-512 which reflects maximum number of
>     CPUs supported by QEMU virt machine.
> ---
>   arch/riscv/Kconfig | 9 ++++++---
>   1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 00fd9c548f26..1823f281069f 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -275,10 +275,13 @@ config SMP
>   	  If you don't know what to do here, say N.
>   
>   config NR_CPUS
> -	int "Maximum number of CPUs (2-32)"
> -	range 2 32
> +	int "Maximum number of CPUs (2-512)"
>   	depends on SMP
> -	default "8"
> +	range 2 512 if !SBI_V01
> +	range 2 32 if SBI_V01 && 32BIT
> +	range 2 64 if SBI_V01 && 64BIT
> +	default "32" if 32BIT
> +	default "64" if 64BIT
>   
>   config HOTPLUG_CPU
>   	bool "Support for hot-pluggable CPUs"

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Atish Patra]

On Thu, Apr 21, 2022 at 12:05 AM Heinrich Schuchardt
<heinrich.schuchardt@canonical.com> wrote:
>
>
>
> On 4/20/22 13:24, Anup Patel wrote:
> > Currently, the range and default value of NR_CPUS is too restrictive
> > for high-end RISC-V systems with large number of HARTs. The latest
> > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > restrictive for QEMU as well. Other major architectures (such as
> > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > value of NR_CPUS.
> >
> > This patch increases NR_CPUS range to 2-512 and default value to
> > XLEN (i.e. 32 for RV32 and 64 for RV64).
> >
> > Signed-off-by: Anup Patel <apatel@ventanamicro.com>
>

This works upto 207 harts in upstream Qemu + OpenSBI. Beyond that,
firmware size becomes > 2MB and PMP protection
kicks in for the kernel boot address (0x80200000)

Here is the OpenSBI boot prints for 256 harts with some hacks around
sanitize_domain to disable sbi_domain_check_addr

Platform Name             : riscv-virtio,qemu
Platform Features         : medeleg
Platform HART Count       : 208
Platform IPI Device       : aclint-mswi
Platform Timer Device     : aclint-mtimer @ 10000000Hz
Platform Console Device   : uart8250
Platform HSM Device       : ---
Platform Reboot Device    : sifive_test
Platform Shutdown Device  : sifive_test
Firmware Base             : 0x80000000
Firmware Size             : 2052 KB ---------->>> Firmware size is
greater than 2MB where
Runtime SBI Version       : 0.3

Domain0 Name              : root
Domain0 Boot HART         : 122
Domain0 HARTs             :
0*,1*,2*,3*,4*,5*,6*,7*,8*,9*,10*,11*,12*,13*,14*,15*,16*,17*,18*,19*,20*,21*,22*,23*,24*,25*,26*,27*,28*,29*,30*,31*,32*,33*,34*,35*,36*,37*,38*,39*,40*,41*,42*,43*,44*,45*,46*,47*,48*,49*,50*,51*,52*,53*,54*,55*,56*,57*,58*,59*,60*,61*,62*,63*,64*,65*,66*,67*,68*,69*,70*,71*,72*,73*,74*,75*,76*,77*,78*,79*,80*,81*,82*,83*,84*,85*,86*,87*,88*,89*,90*,91*,92*,93*,94*,95*,96*,97*,98*,99*,100*,101*,102*,103*,104*,105*,106*,107*,108*,109*,110*,111*,112*,113*,114*,115*,116*,117*,118*,119*,120*,121*,122*,123*,124*,125*,126*,127*,128*,129*,130*,131*,132*,133*,134*,135*,136*,137*,138*,139*,140*,141*,142*,143*,144*,145*,146*,147*,148*,149*,150*,151*,152*,153*,154*,155*,156*,157*,158*,159*,160*,161*,162*,163*,164*,165*,166*,167*,168*,169*,170*,171*,172*,173*,174*,175*,176*,177*,178*,179*,180*,181*,182*,183*,184*,185*,186*,187*,188*,189*,190*,191*,192*,193*,194*,195*,196*,197*,198*,199*,200*,201*,202*,203*,204*,205*,206*,207*
Domain0 Region00          : 0x0000000002000000-0x000000000200ffff (I)
Domain0 Region01          : 0x0000000080000000-0x00000000803fffff ()
Domain0 Region02          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
Domain0 Next Address      : 0x0000000080200000
Domain0 Next Arg1         : 0x00000000bf000000
Domain0 Next Mode         : S-mode
Domain0 SysReset          : yes

Boot HART ID              : 122
Boot HART Domain          : root
Boot HART ISA             : rv64imafdcsuh
Boot HART Features        : scounteren,mcounteren,time
Boot HART PMP Count       : 16
Boot HART PMP Granularity : 4
Boot HART PMP Address Bits: 54
Boot HART MHPM Count      : 0
Boot HART MIDELEG         : 0x0000000000001666
Boot HART MEDELEG         : 0x0000000000f0b509


> Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
>
> > ---
> > Changes since v2:
> >   - Rebased on Linux-5.18-rc3
> >   - Use a different range when SBI v0.1 is enabled
> > Changes since v1:
> >   - Updated NR_CPUS range to 2-512 which reflects maximum number of
> >     CPUs supported by QEMU virt machine.
> > ---
> >   arch/riscv/Kconfig | 9 ++++++---
> >   1 file changed, 6 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 00fd9c548f26..1823f281069f 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -275,10 +275,13 @@ config SMP
> >         If you don't know what to do here, say N.
> >
> >   config NR_CPUS
> > -     int "Maximum number of CPUs (2-32)"
> > -     range 2 32
> > +     int "Maximum number of CPUs (2-512)"
> >       depends on SMP
> > -     default "8"
> > +     range 2 512 if !SBI_V01
> > +     range 2 32 if SBI_V01 && 32BIT
> > +     range 2 64 if SBI_V01 && 64BIT
> > +     default "32" if 32BIT
> > +     default "64" if 64BIT
> >
> >   config HOTPLUG_CPU
> >       bool "Support for hot-pluggable CPUs"



-- 
Regards,
Atish

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Atish Patra]

On Fri, Apr 22, 2022 at 5:16 PM Atish Patra <atishp@atishpatra.org> wrote:
>
> On Thu, Apr 21, 2022 at 12:05 AM Heinrich Schuchardt
> <heinrich.schuchardt@canonical.com> wrote:
> >
> >
> >
> > On 4/20/22 13:24, Anup Patel wrote:
> > > Currently, the range and default value of NR_CPUS is too restrictive
> > > for high-end RISC-V systems with large number of HARTs. The latest
> > > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > > restrictive for QEMU as well. Other major architectures (such as
> > > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > > value of NR_CPUS.
> > >
> > > This patch increases NR_CPUS range to 2-512 and default value to
> > > XLEN (i.e. 32 for RV32 and 64 for RV64).
> > >
> > > Signed-off-by: Anup Patel <apatel@ventanamicro.com>
> >
>
> This works upto 207 harts in upstream Qemu + OpenSBI. Beyond that,
> firmware size becomes > 2MB and PMP protection
> kicks in for the kernel boot address (0x80200000)
>

Just wanted to clarify that this is as per the current configuration
in OpenSBI. The boot address
is configured to be 0x80000000 for the generic platform. Thus, it can
support a maximum of 207 harts.

If anybody wishes to boot more than that, the boot address should be
changed to a valid DRAM address such that
it doesn't step on kernel/DT or other resident boot loaders (e.g. U-Boot).

It should be changed per platform (if supporting more than 128 harts)
rather than changing
to a common address for all platforms.

> Here is the OpenSBI boot prints for 256 harts with some hacks around
> sanitize_domain to disable sbi_domain_check_addr
>
> Platform Name             : riscv-virtio,qemu
> Platform Features         : medeleg
> Platform HART Count       : 208
> Platform IPI Device       : aclint-mswi
> Platform Timer Device     : aclint-mtimer @ 10000000Hz
> Platform Console Device   : uart8250
> Platform HSM Device       : ---
> Platform Reboot Device    : sifive_test
> Platform Shutdown Device  : sifive_test
> Firmware Base             : 0x80000000
> Firmware Size             : 2052 KB ---------->>> Firmware size is
> greater than 2MB where
> Runtime SBI Version       : 0.3
>
> Domain0 Name              : root
> Domain0 Boot HART         : 122
> Domain0 HARTs             :
> 0*,1*,2*,3*,4*,5*,6*,7*,8*,9*,10*,11*,12*,13*,14*,15*,16*,17*,18*,19*,20*,21*,22*,23*,24*,25*,26*,27*,28*,29*,30*,31*,32*,33*,34*,35*,36*,37*,38*,39*,40*,41*,42*,43*,44*,45*,46*,47*,48*,49*,50*,51*,52*,53*,54*,55*,56*,57*,58*,59*,60*,61*,62*,63*,64*,65*,66*,67*,68*,69*,70*,71*,72*,73*,74*,75*,76*,77*,78*,79*,80*,81*,82*,83*,84*,85*,86*,87*,88*,89*,90*,91*,92*,93*,94*,95*,96*,97*,98*,99*,100*,101*,102*,103*,104*,105*,106*,107*,108*,109*,110*,111*,112*,113*,114*,115*,116*,117*,118*,119*,120*,121*,122*,123*,124*,125*,126*,127*,128*,129*,130*,131*,132*,133*,134*,135*,136*,137*,138*,139*,140*,141*,142*,143*,144*,145*,146*,147*,148*,149*,150*,151*,152*,153*,154*,155*,156*,157*,158*,159*,160*,161*,162*,163*,164*,165*,166*,167*,168*,169*,170*,171*,172*,173*,174*,175*,176*,177*,178*,179*,180*,181*,182*,183*,184*,185*,186*,187*,188*,189*,190*,191*,192*,193*,194*,195*,196*,197*,198*,199*,200*,201*,202*,203*,204*,205*,206*,207*
> Domain0 Region00          : 0x0000000002000000-0x000000000200ffff (I)
> Domain0 Region01          : 0x0000000080000000-0x00000000803fffff ()
> Domain0 Region02          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
> Domain0 Next Address      : 0x0000000080200000
> Domain0 Next Arg1         : 0x00000000bf000000
> Domain0 Next Mode         : S-mode
> Domain0 SysReset          : yes
>
> Boot HART ID              : 122
> Boot HART Domain          : root
> Boot HART ISA             : rv64imafdcsuh
> Boot HART Features        : scounteren,mcounteren,time
> Boot HART PMP Count       : 16
> Boot HART PMP Granularity : 4
> Boot HART PMP Address Bits: 54
> Boot HART MHPM Count      : 0
> Boot HART MIDELEG         : 0x0000000000001666
> Boot HART MEDELEG         : 0x0000000000f0b509
>
>
> > Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
> >
> > > ---
> > > Changes since v2:
> > >   - Rebased on Linux-5.18-rc3
> > >   - Use a different range when SBI v0.1 is enabled
> > > Changes since v1:
> > >   - Updated NR_CPUS range to 2-512 which reflects maximum number of
> > >     CPUs supported by QEMU virt machine.
> > > ---
> > >   arch/riscv/Kconfig | 9 ++++++---
> > >   1 file changed, 6 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 00fd9c548f26..1823f281069f 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -275,10 +275,13 @@ config SMP
> > >         If you don't know what to do here, say N.
> > >
> > >   config NR_CPUS
> > > -     int "Maximum number of CPUs (2-32)"
> > > -     range 2 32
> > > +     int "Maximum number of CPUs (2-512)"
> > >       depends on SMP
> > > -     default "8"
> > > +     range 2 512 if !SBI_V01
> > > +     range 2 32 if SBI_V01 && 32BIT
> > > +     range 2 64 if SBI_V01 && 64BIT
> > > +     default "32" if 32BIT
> > > +     default "64" if 64BIT
> > >
> > >   config HOTPLUG_CPU
> > >       bool "Support for hot-pluggable CPUs"
>
>
>
> --
> Regards,
> Atish



-- 
Regards,
Atish

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Anup Patel]

+Alistair

On Sat, Apr 23, 2022 at 7:19 AM Atish Patra <atishp@atishpatra.org> wrote:
>
> On Fri, Apr 22, 2022 at 5:16 PM Atish Patra <atishp@atishpatra.org> wrote:
> >
> > On Thu, Apr 21, 2022 at 12:05 AM Heinrich Schuchardt
> > <heinrich.schuchardt@canonical.com> wrote:
> > >
> > >
> > >
> > > On 4/20/22 13:24, Anup Patel wrote:
> > > > Currently, the range and default value of NR_CPUS is too restrictive
> > > > for high-end RISC-V systems with large number of HARTs. The latest
> > > > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > > > restrictive for QEMU as well. Other major architectures (such as
> > > > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > > > value of NR_CPUS.
> > > >
> > > > This patch increases NR_CPUS range to 2-512 and default value to
> > > > XLEN (i.e. 32 for RV32 and 64 for RV64).
> > > >
> > > > Signed-off-by: Anup Patel <apatel@ventanamicro.com>
> > >
> >
> > This works upto 207 harts in upstream Qemu + OpenSBI. Beyond that,
> > firmware size becomes > 2MB and PMP protection
> > kicks in for the kernel boot address (0x80200000)
> >
>
> Just wanted to clarify that this is as per the current configuration
> in OpenSBI. The boot address
> is configured to be 0x80000000 for the generic platform. Thus, it can
> support a maximum of 207 harts.
>
> If anybody wishes to boot more than that, the boot address should be
> changed to a valid DRAM address such that
> it doesn't step on kernel/DT or other resident boot loaders (e.g. U-Boot).
>
> It should be changed per platform (if supporting more than 128 harts)
> rather than changing
> to a common address for all platforms.

The OpenSBI limit of 128 harts was carefully chosen considering the
fact that QEMU by default loads OpenSBI at start of DRAM and kernel
at 2M (for RV64) or 4M (for RV32) offset. This means for RV64 if
OpenSBI is loaded at the start of DRAM then we only have 2M space
for OpenSBI hence the 128 harts limit.

To use more than 128 harts with OpenSBI, we will have to load it
somewhere else (probably in the middle of DRAM) but this will be a
platform specific choice. If a separate LOADER booting stage (e.g.
U-Boot SPL) is available on a platform then it can load OpenSBI
at desired platform specific location.

Overall, this is indeed a platform configuration issue. We will continue
to optimize OpenSBI to support more harts with less RAM usage but
this is a separate topic.

Regards,
Anup

>
> > Here is the OpenSBI boot prints for 256 harts with some hacks around
> > sanitize_domain to disable sbi_domain_check_addr
> >
> > Platform Name             : riscv-virtio,qemu
> > Platform Features         : medeleg
> > Platform HART Count       : 208
> > Platform IPI Device       : aclint-mswi
> > Platform Timer Device     : aclint-mtimer @ 10000000Hz
> > Platform Console Device   : uart8250
> > Platform HSM Device       : ---
> > Platform Reboot Device    : sifive_test
> > Platform Shutdown Device  : sifive_test
> > Firmware Base             : 0x80000000
> > Firmware Size             : 2052 KB ---------->>> Firmware size is
> > greater than 2MB where
> > Runtime SBI Version       : 0.3
> >
> > Domain0 Name              : root
> > Domain0 Boot HART         : 122
> > Domain0 HARTs             :
> > 0*,1*,2*,3*,4*,5*,6*,7*,8*,9*,10*,11*,12*,13*,14*,15*,16*,17*,18*,19*,20*,21*,22*,23*,24*,25*,26*,27*,28*,29*,30*,31*,32*,33*,34*,35*,36*,37*,38*,39*,40*,41*,42*,43*,44*,45*,46*,47*,48*,49*,50*,51*,52*,53*,54*,55*,56*,57*,58*,59*,60*,61*,62*,63*,64*,65*,66*,67*,68*,69*,70*,71*,72*,73*,74*,75*,76*,77*,78*,79*,80*,81*,82*,83*,84*,85*,86*,87*,88*,89*,90*,91*,92*,93*,94*,95*,96*,97*,98*,99*,100*,101*,102*,103*,104*,105*,106*,107*,108*,109*,110*,111*,112*,113*,114*,115*,116*,117*,118*,119*,120*,121*,122*,123*,124*,125*,126*,127*,128*,129*,130*,131*,132*,133*,134*,135*,136*,137*,138*,139*,140*,141*,142*,143*,144*,145*,146*,147*,148*,149*,150*,151*,152*,153*,154*,155*,156*,157*,158*,159*,160*,161*,162*,163*,164*,165*,166*,167*,168*,169*,170*,171*,172*,173*,174*,175*,176*,177*,178*,179*,180*,181*,182*,183*,184*,185*,186*,187*,188*,189*,190*,191*,192*,193*,194*,195*,196*,197*,198*,199*,200*,201*,202*,203*,204*,205*,206*,207*
> > Domain0 Region00          : 0x0000000002000000-0x000000000200ffff (I)
> > Domain0 Region01          : 0x0000000080000000-0x00000000803fffff ()
> > Domain0 Region02          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
> > Domain0 Next Address      : 0x0000000080200000
> > Domain0 Next Arg1         : 0x00000000bf000000
> > Domain0 Next Mode         : S-mode
> > Domain0 SysReset          : yes
> >
> > Boot HART ID              : 122
> > Boot HART Domain          : root
> > Boot HART ISA             : rv64imafdcsuh
> > Boot HART Features        : scounteren,mcounteren,time
> > Boot HART PMP Count       : 16
> > Boot HART PMP Granularity : 4
> > Boot HART PMP Address Bits: 54
> > Boot HART MHPM Count      : 0
> > Boot HART MIDELEG         : 0x0000000000001666
> > Boot HART MEDELEG         : 0x0000000000f0b509
> >
> >
> > > Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
> > >
> > > > ---
> > > > Changes since v2:
> > > >   - Rebased on Linux-5.18-rc3
> > > >   - Use a different range when SBI v0.1 is enabled
> > > > Changes since v1:
> > > >   - Updated NR_CPUS range to 2-512 which reflects maximum number of
> > > >     CPUs supported by QEMU virt machine.
> > > > ---
> > > >   arch/riscv/Kconfig | 9 ++++++---
> > > >   1 file changed, 6 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index 00fd9c548f26..1823f281069f 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -275,10 +275,13 @@ config SMP
> > > >         If you don't know what to do here, say N.
> > > >
> > > >   config NR_CPUS
> > > > -     int "Maximum number of CPUs (2-32)"
> > > > -     range 2 32
> > > > +     int "Maximum number of CPUs (2-512)"
> > > >       depends on SMP
> > > > -     default "8"
> > > > +     range 2 512 if !SBI_V01
> > > > +     range 2 32 if SBI_V01 && 32BIT
> > > > +     range 2 64 if SBI_V01 && 64BIT
> > > > +     default "32" if 32BIT
> > > > +     default "64" if 64BIT
> > > >
> > > >   config HOTPLUG_CPU
> > > >       bool "Support for hot-pluggable CPUs"
> >
> >
> >
> > --
> > Regards,
> > Atish
>
>
>
> --
> Regards,
> Atish

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Anup Patel]

Hi Palmer,

On Wed, Apr 20, 2022 at 4:54 PM Anup Patel <apatel@ventanamicro.com> wrote:
>
> Currently, the range and default value of NR_CPUS is too restrictive
> for high-end RISC-V systems with large number of HARTs. The latest
> QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> restrictive for QEMU as well. Other major architectures (such as
> ARM64, x86_64, MIPS, etc) have a much higher range and default
> value of NR_CPUS.
>
> This patch increases NR_CPUS range to 2-512 and default value to
> XLEN (i.e. 32 for RV32 and 64 for RV64).
>
> Signed-off-by: Anup Patel <apatel@ventanamicro.com>

Can this PATCH be considered for 5.19 ?

Thanks,
Anup

> ---
> Changes since v2:
>  - Rebased on Linux-5.18-rc3
>  - Use a different range when SBI v0.1 is enabled
> Changes since v1:
>  - Updated NR_CPUS range to 2-512 which reflects maximum number of
>    CPUs supported by QEMU virt machine.
> ---
>  arch/riscv/Kconfig | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 00fd9c548f26..1823f281069f 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -275,10 +275,13 @@ config SMP
>           If you don't know what to do here, say N.
>
>  config NR_CPUS
> -       int "Maximum number of CPUs (2-32)"
> -       range 2 32
> +       int "Maximum number of CPUs (2-512)"
>         depends on SMP
> -       default "8"
> +       range 2 512 if !SBI_V01
> +       range 2 32 if SBI_V01 && 32BIT
> +       range 2 64 if SBI_V01 && 64BIT
> +       default "32" if 32BIT
> +       default "64" if 64BIT
>
>  config HOTPLUG_CPU
>         bool "Support for hot-pluggable CPUs"
> --
> 2.25.1
>

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Atish Patra]

On Tue, May 24, 2022 at 5:08 AM Anup Patel <anup@brainfault.org> wrote:
>
> Hi Palmer,
>
> On Wed, Apr 20, 2022 at 4:54 PM Anup Patel <apatel@ventanamicro.com> wrote:
> >
> > Currently, the range and default value of NR_CPUS is too restrictive
> > for high-end RISC-V systems with large number of HARTs. The latest
> > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > restrictive for QEMU as well. Other major architectures (such as
> > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > value of NR_CPUS.
> >
> > This patch increases NR_CPUS range to 2-512 and default value to
> > XLEN (i.e. 32 for RV32 and 64 for RV64).
> >
> > Signed-off-by: Anup Patel <apatel@ventanamicro.com>
>
> Can this PATCH be considered for 5.19 ?
>
> Thanks,
> Anup
>
> > ---
> > Changes since v2:
> >  - Rebased on Linux-5.18-rc3
> >  - Use a different range when SBI v0.1 is enabled
> > Changes since v1:
> >  - Updated NR_CPUS range to 2-512 which reflects maximum number of
> >    CPUs supported by QEMU virt machine.
> > ---
> >  arch/riscv/Kconfig | 9 ++++++---
> >  1 file changed, 6 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 00fd9c548f26..1823f281069f 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -275,10 +275,13 @@ config SMP
> >           If you don't know what to do here, say N.
> >
> >  config NR_CPUS
> > -       int "Maximum number of CPUs (2-32)"
> > -       range 2 32
> > +       int "Maximum number of CPUs (2-512)"
> >         depends on SMP
> > -       default "8"
> > +       range 2 512 if !SBI_V01
> > +       range 2 32 if SBI_V01 && 32BIT
> > +       range 2 64 if SBI_V01 && 64BIT
> > +       default "32" if 32BIT
> > +       default "64" if 64BIT
> >
> >  config HOTPLUG_CPU
> >         bool "Support for hot-pluggable CPUs"
> > --
> > 2.25.1
> >


Ping ?
It would be useful to include this patch sooner than later to enable
high HART count testing by default.


-- 
Regards,
Atish

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Increase range and default value of NR_CPUS

[Palmer Dabbelt]

On Thu, 18 Aug 2022 14:29:03 PDT (-0700), atishp@atishpatra.org wrote:
> On Tue, May 24, 2022 at 5:08 AM Anup Patel <anup@brainfault.org> wrote:
>>
>> Hi Palmer,
>>
>> On Wed, Apr 20, 2022 at 4:54 PM Anup Patel <apatel@ventanamicro.com> wrote:
>> >
>> > Currently, the range and default value of NR_CPUS is too restrictive
>> > for high-end RISC-V systems with large number of HARTs. The latest
>> > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
>> > restrictive for QEMU as well. Other major architectures (such as
>> > ARM64, x86_64, MIPS, etc) have a much higher range and default
>> > value of NR_CPUS.
>> >
>> > This patch increases NR_CPUS range to 2-512 and default value to
>> > XLEN (i.e. 32 for RV32 and 64 for RV64).
>> >
>> > Signed-off-by: Anup Patel <apatel@ventanamicro.com>
>>
>> Can this PATCH be considered for 5.19 ?
>>
>> Thanks,
>> Anup
>>
>> > ---
>> > Changes since v2:
>> >  - Rebased on Linux-5.18-rc3
>> >  - Use a different range when SBI v0.1 is enabled
>> > Changes since v1:
>> >  - Updated NR_CPUS range to 2-512 which reflects maximum number of
>> >    CPUs supported by QEMU virt machine.
>> > ---
>> >  arch/riscv/Kconfig | 9 ++++++---
>> >  1 file changed, 6 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> > index 00fd9c548f26..1823f281069f 100644
>> > --- a/arch/riscv/Kconfig
>> > +++ b/arch/riscv/Kconfig
>> > @@ -275,10 +275,13 @@ config SMP
>> >           If you don't know what to do here, say N.
>> >
>> >  config NR_CPUS
>> > -       int "Maximum number of CPUs (2-32)"
>> > -       range 2 32
>> > +       int "Maximum number of CPUs (2-512)"
>> >         depends on SMP
>> > -       default "8"
>> > +       range 2 512 if !SBI_V01
>> > +       range 2 32 if SBI_V01 && 32BIT
>> > +       range 2 64 if SBI_V01 && 64BIT
>> > +       default "32" if 32BIT
>> > +       default "64" if 64BIT
>> >
>> >  config HOTPLUG_CPU
>> >         bool "Support for hot-pluggable CPUs"
>> > --
>> > 2.25.1
>> >
>
>
> Ping ?
> It would be useful to include this patch sooner than later to enable
> high HART count testing by default.

Ya, I think that's reasonable: the higher CPU counts have found a bunch 
of issues, but they're not really Linux bugs and stopping folks from 
running them is just going to stop those bugs from being fixed.  It 
seems like these higher default NR_CPUS are stable on smaller systems so 
that shouldn't hurt anything.

I'm still getting a bunch of issues when trying to run the larger CPU 
count systems in QEMU, but I think it's OK just assuming those are 
long-tail issues that would manifest anyway and are just more likely 
with the higher core counts.

So I've got this on for-next, under the rationale that the new default 
CPU counts are safe.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv