Re: [PATCH RFC] x86: Reduce MAX_LOCAL_APIC and MAX_IO_APICS

[Jiang Liu <jiang.liu@linux.intel.com>]

On 2015/9/26 3:48, Denys Vlasenko wrote:
> Before this change MAX_LOCAL_APIC had the fixed value of 32*1024.
> Such a big value causes several data arrays to be quite oversized:
> 
> phys_cpu_present_map is 4 kbytes (one bit per apic id),
> __apicid_to_node[] is 64 kbytes,
> apic_version[] is 128 kbytes.
> 
> On "usual" systems, APIC ids simply go from zero
> to maximum logical CPU number, mirroring CPU ids.
> 
> On broken and unusual multi-socket systems
> APIC ids can be non-contiguous.
> 
> This patch changes MAX_LOCAL_APIC definition as follows:
> 
>  = It is guaranteed to be at least 16.
>  = If NR_CPUS > 16, then it's equal to NR_CPUS.
>  = A new CONFIG_MAX_LAPIC_ID can be used to increase it
>    (but not decrease).
> 
> MAX_IO_APICS was 128. This is a bit large too, making,
> for example, ioapics[] array 9216 bytes big.
> 
> After this patch, MAX_IO_APICS is at least 8, at most 128.
> If NR_CPUS is in this range, then MAX_IO_APICS = NR_CPUS.
> 
> apic_version[] array is changed from int to u8 -
> APIC version values as of year 2015 are no larger than 0x1f
> on all known CPUs.
> 
> A bit of code added to ensure that the statement
> 	apic_version[apicid] = version;
> in generic_processor_info() is safe wrt bad values in both
> 'apicid' and 'version' variables.
> 
> This change reduces NR_CPUS=64 kernel's data size by 204661 bytes:
> 
>     text     data      bss       dec     hex filename
> 91353669 13825744 19021824 124201237 7672915 vmlinux.before
> 91353680 13760336 18882560 123996576 76409a0 vmlinux
> 
> Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com>
> CC: Ingo Molnar <mingo@kernel.org>
> CC: Jiang Liu <jiang.liu@linux.intel.com>
> CC: Thomas Gleixner <tglx@linutronix.de>
> CC: Len Brown <len.brown@intel.com>
> CC: x86@kernel.org
> CC: linux-kernel@vger.kernel.org
> ---
>  arch/x86/Kconfig               | 11 +++++++++++
>  arch/x86/include/asm/apicdef.h | 23 +++++++++++++++++------
>  arch/x86/include/asm/mpspec.h  |  2 +-
>  arch/x86/kernel/apic/apic.c    | 19 ++++++++++++++++++-
>  4 files changed, 47 insertions(+), 8 deletions(-)
> 
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 328c835..9e7c4c1 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -872,6 +872,17 @@ config NR_CPUS
>  	  This is purely to save memory - each supported CPU adds
>  	  approximately eight kilobytes to the kernel image.
>  
> +config MAX_LAPIC_ID
> +	int "Maximum APIC ID"
> +	range 8 32768
> +	default "8"
> +	---help---
> +	  Use this option to set maximum allowed Local APIC ID higher than
> +	  maximum number of CPUs. This may be necessary for machines
> +	  with large number of processor sockets and non-contiguous
> +	  LAPIC numbering.
> +	  This setting will be automatically rounded up, if necessary.
> +
>  config SCHED_SMT
>  	bool "SMT (Hyperthreading) scheduler support"
>  	depends on SMP
> diff --git a/arch/x86/include/asm/apicdef.h b/arch/x86/include/asm/apicdef.h
> index c46bb99..64e2476 100644
> --- a/arch/x86/include/asm/apicdef.h
> +++ b/arch/x86/include/asm/apicdef.h
> @@ -147,15 +147,26 @@
>  #define XAPIC_ENABLE	(1UL << 11)
>  #define X2APIC_ENABLE	(1UL << 10)
>  
> -#ifdef CONFIG_X86_32
> -# define MAX_IO_APICS 64
> -# define MAX_LOCAL_APIC 256
> -#else
> -# define MAX_IO_APICS 128
> -# define MAX_LOCAL_APIC 32768
> +/*
> + * Allow non-contiguous APIC IDs for small machines:
> + * APIC ids 0..15 are valid in any config.
> + * Typical SMP machines have contiguous APIC IDs: 0..NR_CPUS-1.
> + * CONFIG_MAX_LAPIC_ID can override.
> + */
> +#define MAX_LOCAL_APIC (NR_CPUS < 16 ? 16 : NR_CPUS)
> +#if MAX_LOCAL_APIC < CONFIG_MAX_LAPIC_ID
> +# undef  MAX_LOCAL_APIC
> +# define MAX_LOCAL_APIC CONFIG_MAX_LAPIC_ID
>  #endif
>  
>  /*
> + * Minimum is 8.
> + * For largish NR_CPUS, we expect to have no more IOAPICs than CPUs.
> + * No matter how large NR_CPUS is, max is 128.
> + */
> +#define MAX_IO_APICS (NR_CPUS < 8 ? 8 : NR_CPUS < 128 ? NR_CPUS : 128)
This is a little risky. For example, a typical eight-socket Intel
platform will have nine IOAPICs. IO devices may get inaccessible
if some IOAPICs are ignored due to MAX_IO_APICS limitation. It's
a surprising if IO devices get lost if user runs a kernel built with
low NR_CPUS.
Thanks!
Gerry