Re: [PATCH V2 4/8] MIPS: Add NUMA support for Loongson-3

[Ralf Baechle <ralf@linux-mips.org>]

On Sun, Apr 13, 2014 at 08:24:18AM +0800, Huacai Chen wrote:

> Multiple Loongson-3A chips can be interconnected with HT0-bus. This is
> a CC-NUMA system that every chip (node) has its own local memory and
> cache coherency is maintained by hardware. The 64-bit physical memory
> address format is as follows:
> 
> 0x-0000-YZZZ-ZZZZ-ZZZZ
> 
> The high 16 bits should be 0, which means the real physical address
> supported by Loongson-3 is 48-bit. The "Y" bits is the base address of
> each node, which can be also considered as the node-id. The "Z" bits is
> the address offset within a node, which means every node has a 44 bits
> address space.
> 
> Signed-off-by: Huacai Chen <chenhc@lemote.com>

> --- a/arch/mips/Kconfig
> +++ b/arch/mips/Kconfig
> @@ -2233,9 +2233,14 @@ config SYS_SUPPORTS_NUMA
>  	bool
>  
>  config NODES_SHIFT
> -	int
> +	int "Maximum Number of NUMA Nodes Shift"
> +	range 1 10
>  	default "6"
>  	depends on NEED_MULTIPLE_NODES
> +	help
> +	  This option specifies the maximum number of available NUMA nodes
> +	  on the target system. MAX_NUMNODES will be 2^(This value).
> +	  If in doubt, use the default.

I always feel a bit uneasy to present options such as NODES_SHIFT to the
user.

> --- a/arch/mips/include/asm/addrspace.h
> +++ b/arch/mips/include/asm/addrspace.h
> @@ -51,8 +51,14 @@
>   * Returns the physical address of a CKSEGx / XKPHYS address
>   */
>  #define CPHYSADDR(a)		((_ACAST32_(a)) & 0x1fffffff)
> +
> +#ifndef CONFIG_NUMA
>  #define XPHYSADDR(a)		((_ACAST64_(a)) &			\
>  				 _CONST64_(0x000000ffffffffff))
> +#else
> +#define XPHYSADDR(a)		((_ACAST64_(a)) &			\
> +				 _CONST64_(0x0000ffffffffffff))
> +#endif

The mask in XPHYSADDR is a function of the processor architecture, not
imlementation, not NUMA.  The latest version of the MIPS architecture
permits PABITS to be as large as 49 bits, so the mask should be
0x0001ffffffffffff.  Always.

> diff --git a/arch/mips/include/asm/sparsemem.h b/arch/mips/include/asm/sparsemem.h
> index d2da53c..c001a90 100644
> --- a/arch/mips/include/asm/sparsemem.h
> +++ b/arch/mips/include/asm/sparsemem.h
> @@ -11,7 +11,12 @@
>  #else
>  # define SECTION_SIZE_BITS	28
>  #endif
> +
> +#ifdef CONFIG_NUMA
> +#define MAX_PHYSMEM_BITS	48
> +#else
>  #define MAX_PHYSMEM_BITS	35
> +#endif

Essentially the same comment as for XPHYSADDR above.

  Ralf