* Re: [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 14:31 ` [PATCH 1/2] xen: prepare p2m list for memory hotplug Juergen Gross
@ 2015-03-19 15:53 ` Boris Ostrovsky
2015-03-19 17:01 ` Juergen Gross
2015-03-19 16:14 ` Daniel Kiper
` (2 subsequent siblings)
3 siblings, 1 reply; 22+ messages in thread
From: Boris Ostrovsky @ 2015-03-19 15:53 UTC (permalink / raw)
To: Juergen Gross, linux-kernel, xen-devel, konrad.wilk, david.vrabel,
daniel.kiper
On 03/19/2015 10:31 AM, Juergen Gross wrote:
> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
> virtual mapped sparse p2m list") introduced a regression regarding to
> memory hotplug for a pv-domain: as the virtual space for the p2m list
> is allocated for the to be expected memory size of the domain only,
> hotplugged memory above that size will not be usable by the domain.
>
> Correct this by using a configurable size for the p2m list in case of
> memory hotplug enabled (default supported memory size is 512 GB for
> 64 bit domains and 4 GB for 32 bit domains).
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> arch/x86/xen/p2m.c | 13 ++++++++++++-
> drivers/xen/Kconfig | 13 +++++++++++++
> 2 files changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
> index 9f93af5..30e84ae 100644
> --- a/arch/x86/xen/p2m.c
> +++ b/arch/x86/xen/p2m.c
> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
> unsigned long xen_max_p2m_pfn __read_mostly;
> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>
> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> +#ifdef CONFIG_X86_32
> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
Will this work in Kconfig instead?
range 0 64 if CONFIG_X86_32
-boris
> +#endif
> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
> + 1024 * 1024 * 1024 / PAGE_SIZE)))
> +#else
> +#define P2M_LIMIT xen_max_p2m_pfn
> +#endif
> +
> static DEFINE_SPINLOCK(p2m_update_lock);
>
> static unsigned long *p2m_mid_missing_mfn;
> @@ -387,7 +398,7 @@ void __init xen_vmalloc_p2m_tree(void)
> static struct vm_struct vm;
>
> vm.flags = VM_ALLOC;
> - vm.size = ALIGN(sizeof(unsigned long) * xen_max_p2m_pfn,
> + vm.size = ALIGN(sizeof(unsigned long) * P2M_LIMIT,
> PMD_SIZE * PMDS_PER_MID_PAGE);
> vm_area_register_early(&vm, PMD_SIZE * PMDS_PER_MID_PAGE);
> pr_notice("p2m virtual area at %p, size is %lx\n", vm.addr, vm.size);
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index b812462..0a61ddf 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -55,6 +55,19 @@ config XEN_BALLOON_MEMORY_HOTPLUG
>
> In that case step 3 should be omitted.
>
> +config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> + int
> + default 512 if X86_64
> + default 4 if X86_32
> + depends on XEN_HAVE_PVMMU
> + depends on XEN_BALLOON_MEMORY_HOTPLUG
> + help
> + Upper limit in GBs a pv domain can be expanded to using memory
> + hotplug.
> +
> + This value is used to allocate enough space in internal tables needed
> + for physical memory administration.
> +
> config XEN_SCRUB_PAGES
> bool "Scrub pages before returning them to system"
> depends on XEN_BALLOON
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 15:53 ` Boris Ostrovsky
@ 2015-03-19 17:01 ` Juergen Gross
0 siblings, 0 replies; 22+ messages in thread
From: Juergen Gross @ 2015-03-19 17:01 UTC (permalink / raw)
To: Boris Ostrovsky, linux-kernel, xen-devel, konrad.wilk,
david.vrabel, daniel.kiper
On 03/19/2015 04:53 PM, Boris Ostrovsky wrote:
> On 03/19/2015 10:31 AM, Juergen Gross wrote:
>> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
>> virtual mapped sparse p2m list") introduced a regression regarding to
>> memory hotplug for a pv-domain: as the virtual space for the p2m list
>> is allocated for the to be expected memory size of the domain only,
>> hotplugged memory above that size will not be usable by the domain.
>>
>> Correct this by using a configurable size for the p2m list in case of
>> memory hotplug enabled (default supported memory size is 512 GB for
>> 64 bit domains and 4 GB for 32 bit domains).
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>> arch/x86/xen/p2m.c | 13 ++++++++++++-
>> drivers/xen/Kconfig | 13 +++++++++++++
>> 2 files changed, 25 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
>> index 9f93af5..30e84ae 100644
>> --- a/arch/x86/xen/p2m.c
>> +++ b/arch/x86/xen/p2m.c
>> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
>> unsigned long xen_max_p2m_pfn __read_mostly;
>> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>> +#ifdef CONFIG_X86_32
>> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
>
> Will this work in Kconfig instead?
>
> range 0 64 if CONFIG_X86_32
I'll give it a try.
Juergen
>
> -boris
>
>> +#endif
>> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
>> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
>> + 1024 * 1024 * 1024 / PAGE_SIZE)))
>> +#else
>> +#define P2M_LIMIT xen_max_p2m_pfn
>> +#endif
>> +
>> static DEFINE_SPINLOCK(p2m_update_lock);
>> static unsigned long *p2m_mid_missing_mfn;
>> @@ -387,7 +398,7 @@ void __init xen_vmalloc_p2m_tree(void)
>> static struct vm_struct vm;
>> vm.flags = VM_ALLOC;
>> - vm.size = ALIGN(sizeof(unsigned long) * xen_max_p2m_pfn,
>> + vm.size = ALIGN(sizeof(unsigned long) * P2M_LIMIT,
>> PMD_SIZE * PMDS_PER_MID_PAGE);
>> vm_area_register_early(&vm, PMD_SIZE * PMDS_PER_MID_PAGE);
>> pr_notice("p2m virtual area at %p, size is %lx\n", vm.addr,
>> vm.size);
>> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
>> index b812462..0a61ddf 100644
>> --- a/drivers/xen/Kconfig
>> +++ b/drivers/xen/Kconfig
>> @@ -55,6 +55,19 @@ config XEN_BALLOON_MEMORY_HOTPLUG
>> In that case step 3 should be omitted.
>> +config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>> + int
>> + default 512 if X86_64
>> + default 4 if X86_32
>> + depends on XEN_HAVE_PVMMU
>> + depends on XEN_BALLOON_MEMORY_HOTPLUG
>> + help
>> + Upper limit in GBs a pv domain can be expanded to using memory
>> + hotplug.
>> +
>> + This value is used to allocate enough space in internal tables
>> needed
>> + for physical memory administration.
>> +
>> config XEN_SCRUB_PAGES
>> bool "Scrub pages before returning them to system"
>> depends on XEN_BALLOON
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 14:31 ` [PATCH 1/2] xen: prepare p2m list for memory hotplug Juergen Gross
2015-03-19 15:53 ` Boris Ostrovsky
@ 2015-03-19 16:14 ` Daniel Kiper
2015-03-19 17:05 ` Juergen Gross
2015-03-19 16:18 ` David Vrabel
2015-03-19 19:15 ` Paul Bolle
3 siblings, 1 reply; 22+ messages in thread
From: Daniel Kiper @ 2015-03-19 16:14 UTC (permalink / raw)
To: Juergen Gross
Cc: linux-kernel, xen-devel, konrad.wilk, david.vrabel,
boris.ostrovsky
On Thu, Mar 19, 2015 at 03:31:01PM +0100, Juergen Gross wrote:
> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
> virtual mapped sparse p2m list") introduced a regression regarding to
> memory hotplug for a pv-domain: as the virtual space for the p2m list
> is allocated for the to be expected memory size of the domain only,
> hotplugged memory above that size will not be usable by the domain.
>
> Correct this by using a configurable size for the p2m list in case of
> memory hotplug enabled (default supported memory size is 512 GB for
> 64 bit domains and 4 GB for 32 bit domains).
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> arch/x86/xen/p2m.c | 13 ++++++++++++-
> drivers/xen/Kconfig | 13 +++++++++++++
> 2 files changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
> index 9f93af5..30e84ae 100644
> --- a/arch/x86/xen/p2m.c
> +++ b/arch/x86/xen/p2m.c
> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
> unsigned long xen_max_p2m_pfn __read_mostly;
> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>
> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> +#ifdef CONFIG_X86_32
> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
> +#endif
> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
> + 1024 * 1024 * 1024 / PAGE_SIZE)))
> +#else
> +#define P2M_LIMIT xen_max_p2m_pfn
> +#endif
> +
> static DEFINE_SPINLOCK(p2m_update_lock);
>
> static unsigned long *p2m_mid_missing_mfn;
> @@ -387,7 +398,7 @@ void __init xen_vmalloc_p2m_tree(void)
> static struct vm_struct vm;
>
> vm.flags = VM_ALLOC;
> - vm.size = ALIGN(sizeof(unsigned long) * xen_max_p2m_pfn,
> + vm.size = ALIGN(sizeof(unsigned long) * P2M_LIMIT,
> PMD_SIZE * PMDS_PER_MID_PAGE);
What happens when somebody will allocate more memory for guest than
XEN_BALLOON_MEMORY_HOTPLUG_LIMIT? Additionally, I think that we do
not need allocate extra virtual address space if memory hotplug is
disabled.
> vm_area_register_early(&vm, PMD_SIZE * PMDS_PER_MID_PAGE);
> pr_notice("p2m virtual area at %p, size is %lx\n", vm.addr, vm.size);
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index b812462..0a61ddf 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -55,6 +55,19 @@ config XEN_BALLOON_MEMORY_HOTPLUG
>
> In that case step 3 should be omitted.
>
> +config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> + int
> + default 512 if X86_64
> + default 4 if X86_32
> + depends on XEN_HAVE_PVMMU
Do we need dependency on XEN_HAVE_PVMMU?
If yes than maybe "depends on XEN_HAVE_PVMMU && XEN_BALLOON_MEMORY_HOTPLUG".
> + depends on XEN_BALLOON_MEMORY_HOTPLUG
Should not this option be available even when memory hotplug is disabled?
Daniel
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 16:14 ` Daniel Kiper
@ 2015-03-19 17:05 ` Juergen Gross
0 siblings, 0 replies; 22+ messages in thread
From: Juergen Gross @ 2015-03-19 17:05 UTC (permalink / raw)
To: Daniel Kiper
Cc: linux-kernel, xen-devel, konrad.wilk, david.vrabel,
boris.ostrovsky
On 03/19/2015 05:14 PM, Daniel Kiper wrote:
> On Thu, Mar 19, 2015 at 03:31:01PM +0100, Juergen Gross wrote:
>> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
>> virtual mapped sparse p2m list") introduced a regression regarding to
>> memory hotplug for a pv-domain: as the virtual space for the p2m list
>> is allocated for the to be expected memory size of the domain only,
>> hotplugged memory above that size will not be usable by the domain.
>>
>> Correct this by using a configurable size for the p2m list in case of
>> memory hotplug enabled (default supported memory size is 512 GB for
>> 64 bit domains and 4 GB for 32 bit domains).
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>> arch/x86/xen/p2m.c | 13 ++++++++++++-
>> drivers/xen/Kconfig | 13 +++++++++++++
>> 2 files changed, 25 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
>> index 9f93af5..30e84ae 100644
>> --- a/arch/x86/xen/p2m.c
>> +++ b/arch/x86/xen/p2m.c
>> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
>> unsigned long xen_max_p2m_pfn __read_mostly;
>> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>>
>> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>> +#ifdef CONFIG_X86_32
>> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
>> +#endif
>> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
>> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
>> + 1024 * 1024 * 1024 / PAGE_SIZE)))
>> +#else
>> +#define P2M_LIMIT xen_max_p2m_pfn
>> +#endif
>> +
>> static DEFINE_SPINLOCK(p2m_update_lock);
>>
>> static unsigned long *p2m_mid_missing_mfn;
>> @@ -387,7 +398,7 @@ void __init xen_vmalloc_p2m_tree(void)
>> static struct vm_struct vm;
>>
>> vm.flags = VM_ALLOC;
>> - vm.size = ALIGN(sizeof(unsigned long) * xen_max_p2m_pfn,
>> + vm.size = ALIGN(sizeof(unsigned long) * P2M_LIMIT,
>> PMD_SIZE * PMDS_PER_MID_PAGE);
>
> What happens when somebody will allocate more memory for guest than
> XEN_BALLOON_MEMORY_HOTPLUG_LIMIT? Additionally, I think that we do
It will fail to add like it does without this patch.
> not need allocate extra virtual address space if memory hotplug is
> disabled.
As XEN_BALLOON_MEMORY_HOTPLUG_LIMIT depends on
XEN_BALLOON_MEMORY_HOTPLUG which in turn depends on MEMORY_HOTPLUG,
this is already the case.
>
>> vm_area_register_early(&vm, PMD_SIZE * PMDS_PER_MID_PAGE);
>> pr_notice("p2m virtual area at %p, size is %lx\n", vm.addr, vm.size);
>> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
>> index b812462..0a61ddf 100644
>> --- a/drivers/xen/Kconfig
>> +++ b/drivers/xen/Kconfig
>> @@ -55,6 +55,19 @@ config XEN_BALLOON_MEMORY_HOTPLUG
>>
>> In that case step 3 should be omitted.
>>
>> +config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>> + int
>> + default 512 if X86_64
>> + default 4 if X86_32
>> + depends on XEN_HAVE_PVMMU
>
> Do we need dependency on XEN_HAVE_PVMMU?
Yes. Needed only for pv domains.
> If yes than maybe "depends on XEN_HAVE_PVMMU && XEN_BALLOON_MEMORY_HOTPLUG".
Matter of taste, I guess.
>
>> + depends on XEN_BALLOON_MEMORY_HOTPLUG
>
> Should not this option be available even when memory hotplug is disabled?
Why?
Juergen
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 1/2] xen: prepare p2m list for memory hotplug
@ 2015-03-19 17:05 ` Juergen Gross
0 siblings, 0 replies; 22+ messages in thread
From: Juergen Gross @ 2015-03-19 17:05 UTC (permalink / raw)
To: Daniel Kiper; +Cc: david.vrabel, boris.ostrovsky, xen-devel, linux-kernel
On 03/19/2015 05:14 PM, Daniel Kiper wrote:
> On Thu, Mar 19, 2015 at 03:31:01PM +0100, Juergen Gross wrote:
>> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
>> virtual mapped sparse p2m list") introduced a regression regarding to
>> memory hotplug for a pv-domain: as the virtual space for the p2m list
>> is allocated for the to be expected memory size of the domain only,
>> hotplugged memory above that size will not be usable by the domain.
>>
>> Correct this by using a configurable size for the p2m list in case of
>> memory hotplug enabled (default supported memory size is 512 GB for
>> 64 bit domains and 4 GB for 32 bit domains).
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>> arch/x86/xen/p2m.c | 13 ++++++++++++-
>> drivers/xen/Kconfig | 13 +++++++++++++
>> 2 files changed, 25 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
>> index 9f93af5..30e84ae 100644
>> --- a/arch/x86/xen/p2m.c
>> +++ b/arch/x86/xen/p2m.c
>> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
>> unsigned long xen_max_p2m_pfn __read_mostly;
>> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>>
>> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>> +#ifdef CONFIG_X86_32
>> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
>> +#endif
>> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
>> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
>> + 1024 * 1024 * 1024 / PAGE_SIZE)))
>> +#else
>> +#define P2M_LIMIT xen_max_p2m_pfn
>> +#endif
>> +
>> static DEFINE_SPINLOCK(p2m_update_lock);
>>
>> static unsigned long *p2m_mid_missing_mfn;
>> @@ -387,7 +398,7 @@ void __init xen_vmalloc_p2m_tree(void)
>> static struct vm_struct vm;
>>
>> vm.flags = VM_ALLOC;
>> - vm.size = ALIGN(sizeof(unsigned long) * xen_max_p2m_pfn,
>> + vm.size = ALIGN(sizeof(unsigned long) * P2M_LIMIT,
>> PMD_SIZE * PMDS_PER_MID_PAGE);
>
> What happens when somebody will allocate more memory for guest than
> XEN_BALLOON_MEMORY_HOTPLUG_LIMIT? Additionally, I think that we do
It will fail to add like it does without this patch.
> not need allocate extra virtual address space if memory hotplug is
> disabled.
As XEN_BALLOON_MEMORY_HOTPLUG_LIMIT depends on
XEN_BALLOON_MEMORY_HOTPLUG which in turn depends on MEMORY_HOTPLUG,
this is already the case.
>
>> vm_area_register_early(&vm, PMD_SIZE * PMDS_PER_MID_PAGE);
>> pr_notice("p2m virtual area at %p, size is %lx\n", vm.addr, vm.size);
>> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
>> index b812462..0a61ddf 100644
>> --- a/drivers/xen/Kconfig
>> +++ b/drivers/xen/Kconfig
>> @@ -55,6 +55,19 @@ config XEN_BALLOON_MEMORY_HOTPLUG
>>
>> In that case step 3 should be omitted.
>>
>> +config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>> + int
>> + default 512 if X86_64
>> + default 4 if X86_32
>> + depends on XEN_HAVE_PVMMU
>
> Do we need dependency on XEN_HAVE_PVMMU?
Yes. Needed only for pv domains.
> If yes than maybe "depends on XEN_HAVE_PVMMU && XEN_BALLOON_MEMORY_HOTPLUG".
Matter of taste, I guess.
>
>> + depends on XEN_BALLOON_MEMORY_HOTPLUG
>
> Should not this option be available even when memory hotplug is disabled?
Why?
Juergen
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Xen-devel] [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 14:31 ` [PATCH 1/2] xen: prepare p2m list for memory hotplug Juergen Gross
@ 2015-03-19 16:18 ` David Vrabel
2015-03-19 16:14 ` Daniel Kiper
` (2 subsequent siblings)
3 siblings, 0 replies; 22+ messages in thread
From: David Vrabel @ 2015-03-19 16:18 UTC (permalink / raw)
To: Juergen Gross, linux-kernel, xen-devel, konrad.wilk, david.vrabel,
boris.ostrovsky, daniel.kiper
On 19/03/15 14:31, Juergen Gross wrote:
> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
> virtual mapped sparse p2m list") introduced a regression regarding to
> memory hotplug for a pv-domain: as the virtual space for the p2m list
> is allocated for the to be expected memory size of the domain only,
> hotplugged memory above that size will not be usable by the domain.
>
> Correct this by using a configurable size for the p2m list in case of
> memory hotplug enabled (default supported memory size is 512 GB for
> 64 bit domains and 4 GB for 32 bit domains).
[...]
> --- a/arch/x86/xen/p2m.c
> +++ b/arch/x86/xen/p2m.c
> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
> unsigned long xen_max_p2m_pfn __read_mostly;
> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>
> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> +#ifdef CONFIG_X86_32
> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
> +#endif
> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
> + 1024 * 1024 * 1024 / PAGE_SIZE)))
> +#else
> +#define P2M_LIMIT xen_max_p2m_pfn
> +#endif
Can you arrange the #ifdef's to set xen_max_p2m_pfn to the right value
instead of introducing P2M_LIMIT?
David
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [Xen-devel] [PATCH 1/2] xen: prepare p2m list for memory hotplug
@ 2015-03-19 16:18 ` David Vrabel
0 siblings, 0 replies; 22+ messages in thread
From: David Vrabel @ 2015-03-19 16:18 UTC (permalink / raw)
To: Juergen Gross, linux-kernel, xen-devel, konrad.wilk, david.vrabel,
boris.ostrovsky, daniel.kiper
On 19/03/15 14:31, Juergen Gross wrote:
> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
> virtual mapped sparse p2m list") introduced a regression regarding to
> memory hotplug for a pv-domain: as the virtual space for the p2m list
> is allocated for the to be expected memory size of the domain only,
> hotplugged memory above that size will not be usable by the domain.
>
> Correct this by using a configurable size for the p2m list in case of
> memory hotplug enabled (default supported memory size is 512 GB for
> 64 bit domains and 4 GB for 32 bit domains).
[...]
> --- a/arch/x86/xen/p2m.c
> +++ b/arch/x86/xen/p2m.c
> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
> unsigned long xen_max_p2m_pfn __read_mostly;
> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>
> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> +#ifdef CONFIG_X86_32
> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
> +#endif
> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
> + 1024 * 1024 * 1024 / PAGE_SIZE)))
> +#else
> +#define P2M_LIMIT xen_max_p2m_pfn
> +#endif
Can you arrange the #ifdef's to set xen_max_p2m_pfn to the right value
instead of introducing P2M_LIMIT?
David
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [Xen-devel] [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 16:18 ` David Vrabel
(?)
@ 2015-03-19 17:16 ` Juergen Gross
2015-03-19 17:22 ` David Vrabel
-1 siblings, 1 reply; 22+ messages in thread
From: Juergen Gross @ 2015-03-19 17:16 UTC (permalink / raw)
To: David Vrabel, linux-kernel, xen-devel, konrad.wilk,
boris.ostrovsky, daniel.kiper
On 03/19/2015 05:18 PM, David Vrabel wrote:
> On 19/03/15 14:31, Juergen Gross wrote:
>> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
>> virtual mapped sparse p2m list") introduced a regression regarding to
>> memory hotplug for a pv-domain: as the virtual space for the p2m list
>> is allocated for the to be expected memory size of the domain only,
>> hotplugged memory above that size will not be usable by the domain.
>>
>> Correct this by using a configurable size for the p2m list in case of
>> memory hotplug enabled (default supported memory size is 512 GB for
>> 64 bit domains and 4 GB for 32 bit domains).
> [...]
>> --- a/arch/x86/xen/p2m.c
>> +++ b/arch/x86/xen/p2m.c
>> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
>> unsigned long xen_max_p2m_pfn __read_mostly;
>> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>>
>> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>> +#ifdef CONFIG_X86_32
>> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
>> +#endif
>> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
>> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT * \
>> + 1024 * 1024 * 1024 / PAGE_SIZE)))
>> +#else
>> +#define P2M_LIMIT xen_max_p2m_pfn
>> +#endif
>
> Can you arrange the #ifdef's to set xen_max_p2m_pfn to the right value
> instead of introducing P2M_LIMIT?
Hmm, this would require additional checks in setup.c. What about:
#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
#define P2M_LIMIT CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
#else
#define P2M_LIMIT 0
#endif
and do the max(...) calculation in xen_vmalloc_p2m_tree()?
Juergen
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [Xen-devel] [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 17:16 ` Juergen Gross
@ 2015-03-19 17:22 ` David Vrabel
0 siblings, 0 replies; 22+ messages in thread
From: David Vrabel @ 2015-03-19 17:22 UTC (permalink / raw)
To: Juergen Gross, linux-kernel, xen-devel, konrad.wilk,
boris.ostrovsky, daniel.kiper
On 19/03/15 17:16, Juergen Gross wrote:
> On 03/19/2015 05:18 PM, David Vrabel wrote:
>> On 19/03/15 14:31, Juergen Gross wrote:
>>> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
>>> virtual mapped sparse p2m list") introduced a regression regarding to
>>> memory hotplug for a pv-domain: as the virtual space for the p2m list
>>> is allocated for the to be expected memory size of the domain only,
>>> hotplugged memory above that size will not be usable by the domain.
>>>
>>> Correct this by using a configurable size for the p2m list in case of
>>> memory hotplug enabled (default supported memory size is 512 GB for
>>> 64 bit domains and 4 GB for 32 bit domains).
>> [...]
>>> --- a/arch/x86/xen/p2m.c
>>> +++ b/arch/x86/xen/p2m.c
>>> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
>>> unsigned long xen_max_p2m_pfn __read_mostly;
>>> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>>>
>>> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>>> +#ifdef CONFIG_X86_32
>>> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
>>> +#endif
>>> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
>>> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>>> * \
>>> + 1024 * 1024 * 1024 / PAGE_SIZE)))
>>> +#else
>>> +#define P2M_LIMIT xen_max_p2m_pfn
>>> +#endif
>>
>> Can you arrange the #ifdef's to set xen_max_p2m_pfn to the right value
>> instead of introducing P2M_LIMIT?
>
> Hmm, this would require additional checks in setup.c. What about:
>
> #ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> #define P2M_LIMIT CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> #else
> #define P2M_LIMIT 0
> #endif
>
> and do the max(...) calculation in xen_vmalloc_p2m_tree()?
Yes, this is fine.
David
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [Xen-devel] [PATCH 1/2] xen: prepare p2m list for memory hotplug
@ 2015-03-19 17:22 ` David Vrabel
0 siblings, 0 replies; 22+ messages in thread
From: David Vrabel @ 2015-03-19 17:22 UTC (permalink / raw)
To: Juergen Gross, linux-kernel, xen-devel, konrad.wilk,
boris.ostrovsky, daniel.kiper
On 19/03/15 17:16, Juergen Gross wrote:
> On 03/19/2015 05:18 PM, David Vrabel wrote:
>> On 19/03/15 14:31, Juergen Gross wrote:
>>> Commit 054954eb051f35e74b75a566a96fe756015352c8 ("xen: switch to linear
>>> virtual mapped sparse p2m list") introduced a regression regarding to
>>> memory hotplug for a pv-domain: as the virtual space for the p2m list
>>> is allocated for the to be expected memory size of the domain only,
>>> hotplugged memory above that size will not be usable by the domain.
>>>
>>> Correct this by using a configurable size for the p2m list in case of
>>> memory hotplug enabled (default supported memory size is 512 GB for
>>> 64 bit domains and 4 GB for 32 bit domains).
>> [...]
>>> --- a/arch/x86/xen/p2m.c
>>> +++ b/arch/x86/xen/p2m.c
>>> @@ -91,6 +91,17 @@ EXPORT_SYMBOL_GPL(xen_p2m_size);
>>> unsigned long xen_max_p2m_pfn __read_mostly;
>>> EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
>>>
>>> +#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>>> +#ifdef CONFIG_X86_32
>>> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
>>> +#endif
>>> +#define P2M_LIMIT max(xen_max_p2m_pfn, \
>>> + ((unsigned long)((u64)CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
>>> * \
>>> + 1024 * 1024 * 1024 / PAGE_SIZE)))
>>> +#else
>>> +#define P2M_LIMIT xen_max_p2m_pfn
>>> +#endif
>>
>> Can you arrange the #ifdef's to set xen_max_p2m_pfn to the right value
>> instead of introducing P2M_LIMIT?
>
> Hmm, this would require additional checks in setup.c. What about:
>
> #ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> #define P2M_LIMIT CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> #else
> #define P2M_LIMIT 0
> #endif
>
> and do the max(...) calculation in xen_vmalloc_p2m_tree()?
Yes, this is fine.
David
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 1/2] xen: prepare p2m list for memory hotplug
2015-03-19 14:31 ` [PATCH 1/2] xen: prepare p2m list for memory hotplug Juergen Gross
` (2 preceding siblings ...)
2015-03-19 16:18 ` David Vrabel
@ 2015-03-19 19:15 ` Paul Bolle
3 siblings, 0 replies; 22+ messages in thread
From: Paul Bolle @ 2015-03-19 19:15 UTC (permalink / raw)
To: Juergen Gross
Cc: linux-kernel, xen-devel, konrad.wilk, david.vrabel,
boris.ostrovsky, daniel.kiper
On Thu, 2015-03-19 at 15:31 +0100, Juergen Gross wrote:
> --- a/arch/x86/xen/p2m.c
> +++ b/arch/x86/xen/p2m.c
> +#ifdef CONFIG_X86_32
> +BUILD_BUG_ON_MSG(CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT > 64)
> +#endif
I assume BUILD_BUG_ON_MSG() aborts the build.
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> +config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
> + int
> + default 512 if X86_64
> + default 4 if X86_32
> + depends on XEN_HAVE_PVMMU
> + depends on XEN_BALLOON_MEMORY_HOTPLUG
> + help
> + Upper limit in GBs a pv domain can be expanded to using memory
> + hotplug.
> +
> + This value is used to allocate enough space in internal tables needed
> + for physical memory administration.
> +
I think adding a
range 1 64 if X86_32
would allow to drop the BUILD_BUG_ON_MSG(). (I haven't tested this so
you're allowed to bark at me if this ends up wasting your time.)
Paul Bolle
^ permalink raw reply [flat|nested] 22+ messages in thread