[PATCH v2] xen: arm32: reduce default size of the xenheap

[PATCH v2] xen: arm32: reduce default size of the xenheap

[Ian Campbell]

... and make it tunable via the command line.

1/8 of RAM is 128M on a 1GB system and 256M on a 2GB system etc,
which is a lot. 1/32 of RAM seems more reasonable. Also drop the
minimum to 32M.

Leave the maximum at 1GB.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Jintack Lim <jintack@cs.columbia.edu>
Cc: Jan Beulich <JBeulich@suse.com>
---
v2:
 - Use xenheap_megabytes as the option, which is what older x86 Xen
   used.

I'd like to backport at least the command line option to 4.5. Reducing
the default heap size is a bit border line but I'm inclined to take
it.
---
 docs/misc/xen-command-line.markdown |    8 ++++++++
 xen/arch/arm/setup.c                |   25 ++++++++++++++++++-------
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index bc316be..0a99d1a 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -237,6 +237,14 @@ and not running softirqs. Reduce this if softirqs are not being run frequently
 enough. Setting this to a high value may cause boot failure, particularly if
 the NMI watchdog is also enabled.
 
+### xenheap\_megabytes (arm32)
+> `= <size>`
+
+> Default: `1/32 RAM`
+
+Amount of RAM to set aside for the Xenheap. By default 1/32 of the RAM
+up to a maximum of 1GB and with a minimum of 32M.
+
 ### clocksource
 > `= pit | hpet | acpi`
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index a916ca6..73691c0 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -50,6 +50,11 @@ struct bootinfo __initdata bootinfo;
 
 struct cpuinfo_arm __read_mostly boot_cpu_data;
 
+#ifdef CONFIG_ARM_32
+static unsigned long opt_xenheap_megabytes __initdata;
+integer_param("xenheap_megabytes", opt_xenheap_megabytes);
+#endif
+
 static __used void init_done(void)
 {
     free_init_memory();
@@ -501,16 +506,21 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
      *
      *  - must be 32 MiB aligned
      *  - must not include Xen itself or the boot modules
-     *  - must be at most 1GB or 1/8 the total RAM in the system if less
-     *  - must be at least 128M
+     *  - must be at most 1GB or 1/32 the total RAM in the system if less
+     *  - must be at least 32M
      *
      * We try to allocate the largest xenheap possible within these
      * constraints.
      */
     heap_pages = ram_pages;
-    xenheap_pages = (heap_pages/8 + 0x1fffUL) & ~0x1fffUL;
-    xenheap_pages = max(xenheap_pages, 128UL<<(20-PAGE_SHIFT));
-    xenheap_pages = min(xenheap_pages, 1UL<<(30-PAGE_SHIFT));
+    if ( opt_xenheap_megabytes )
+        xenheap_pages = opt_xenheap_megabytes << (20-PAGE_SHIFT);
+    else
+    {
+        xenheap_pages = (heap_pages/32 + 0x1fffUL) & ~0x1fffUL;
+        xenheap_pages = max(xenheap_pages, 32UL<<(20-PAGE_SHIFT));
+        xenheap_pages = min(xenheap_pages, 1UL<<(30-PAGE_SHIFT));
+    }
 
     do
     {
@@ -528,8 +538,9 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
 
     domheap_pages = heap_pages - xenheap_pages;
 
-    printk("Xen heap: %"PRIpaddr"-%"PRIpaddr" (%lu pages)\n",
-            e - (pfn_to_paddr(xenheap_pages)), e, xenheap_pages);
+    printk("Xen heap: %"PRIpaddr"-%"PRIpaddr" (%lu pages%s)\n",
+           e - (pfn_to_paddr(xenheap_pages)), e, xenheap_pages,
+           opt_xenheap_megabytes ? ", from command-line" : "");
     printk("Dom heap: %lu pages\n", domheap_pages);
 
     setup_xenheap_mappings((e >> PAGE_SHIFT) - xenheap_pages, xenheap_pages);
-- 
1.7.10.4

Re: [PATCH v2] xen: arm32: reduce default size of the xenheap

[Jan Beulich]

>>> On 05.02.15 at 11:52, <ian.campbell@citrix.com> wrote:
> ... and make it tunable via the command line.
> 
> 1/8 of RAM is 128M on a 1GB system and 256M on a 2GB system etc,
> which is a lot. 1/32 of RAM seems more reasonable. Also drop the
> minimum to 32M.
> 
> Leave the maximum at 1GB.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Jintack Lim <jintack@cs.columbia.edu>
> Cc: Jan Beulich <JBeulich@suse.com>
> ---
> v2:
>  - Use xenheap_megabytes as the option, which is what older x86 Xen
>    used.
> 
> I'd like to backport at least the command line option to 4.5. Reducing
> the default heap size is a bit border line but I'm inclined to take
> it.

Since one can override it in the worst case, I think that's acceptable.

Jan

Re: [PATCH v2] xen: arm32: reduce default size of the xenheap

[Julien Grall]

Hi Ian,

On 05/02/2015 18:52, Ian Campbell wrote:
> ... and make it tunable via the command line.
>
> 1/8 of RAM is 128M on a 1GB system and 256M on a 2GB system etc,
> which is a lot. 1/32 of RAM seems more reasonable. Also drop the
> minimum to 32M.
>
> Leave the maximum at 1GB.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Jintack Lim <jintack@cs.columbia.edu>
> Cc: Jan Beulich <JBeulich@suse.com>
> ---
> v2:
>   - Use xenheap_megabytes as the option, which is what older x86 Xen
>     used.
>
> I'd like to backport at least the command line option to 4.5. Reducing
> the default heap size is a bit border line but I'm inclined to take
> it.

I don't think it will introduce performance regression. So I don't mind 
if you backport the whole patch.

> ---
>   docs/misc/xen-command-line.markdown |    8 ++++++++
>   xen/arch/arm/setup.c                |   25 ++++++++++++++++++-------
>   2 files changed, 26 insertions(+), 7 deletions(-)
>
> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
> index bc316be..0a99d1a 100644
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -237,6 +237,14 @@ and not running softirqs. Reduce this if softirqs are not being run frequently
>   enough. Setting this to a high value may cause boot failure, particularly if
>   the NMI watchdog is also enabled.
>
> +### xenheap\_megabytes (arm32)
> +> `= <size>`
> +
> +> Default: `1/32 RAM`
> +
> +Amount of RAM to set aside for the Xenheap. By default 1/32 of the RAM
> +up to a maximum of 1GB and with a minimum of 32M.
> +
>   ### clocksource
>   > `= pit | hpet | acpi`
>
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index a916ca6..73691c0 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -50,6 +50,11 @@ struct bootinfo __initdata bootinfo;
>
>   struct cpuinfo_arm __read_mostly boot_cpu_data;
>
> +#ifdef CONFIG_ARM_32
> +static unsigned long opt_xenheap_megabytes __initdata;
> +integer_param("xenheap_megabytes", opt_xenheap_megabytes);
> +#endif
> +
>   static __used void init_done(void)
>   {
>       free_init_memory();
> @@ -501,16 +506,21 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
>        *
>        *  - must be 32 MiB aligned
>        *  - must not include Xen itself or the boot modules
> -     *  - must be at most 1GB or 1/8 the total RAM in the system if less
> -     *  - must be at least 128M
> +     *  - must be at most 1GB or 1/32 the total RAM in the system if less
> +     *  - must be at least 32M
>        *
>        * We try to allocate the largest xenheap possible within these
>        * constraints.
>        */
>       heap_pages = ram_pages;
> -    xenheap_pages = (heap_pages/8 + 0x1fffUL) & ~0x1fffUL;
> -    xenheap_pages = max(xenheap_pages, 128UL<<(20-PAGE_SHIFT));
> -    xenheap_pages = min(xenheap_pages, 1UL<<(30-PAGE_SHIFT));
> +    if ( opt_xenheap_megabytes )

If the user requests a xenheap of 0MB, we will use the default size, 
right? It may be worth to explain this case.

Also with the algorithm to find a range, the Xen heap may be smaller. I 
would explain that too.

> +        xenheap_pages = opt_xenheap_megabytes << (20-PAGE_SHIFT);
> +    else
> +    {
> +        xenheap_pages = (heap_pages/32 + 0x1fffUL) & ~0x1fffUL;
> +        xenheap_pages = max(xenheap_pages, 32UL<<(20-PAGE_SHIFT));
> +        xenheap_pages = min(xenheap_pages, 1UL<<(30-PAGE_SHIFT));
> +    }
>       do
>       {

You seem to have forgotten to update the condition in the do/while:

xenheap_pages > 128 << (20 - PAGE_SHIFT);

Regards,

-- 
Julien Grall

Re: [PATCH v2] xen: arm32: reduce default size of the xenheap

[Ian Campbell]

On Thu, 2015-02-05 at 22:01 +0800, Julien Grall wrote:
> If the user requests a xenheap of 0MB, we will use the default size, 
> right? It may be worth to explain this case.

I think it's pretty generally understood that 0 generally means the
default, unless otherwise specified.

> Also with the algorithm to find a range, the Xen heap may be smaller. I 
> would explain that too.

True.

> > +        xenheap_pages = opt_xenheap_megabytes << (20-PAGE_SHIFT);
> > +    else
> > +    {
> > +        xenheap_pages = (heap_pages/32 + 0x1fffUL) & ~0x1fffUL;
> > +        xenheap_pages = max(xenheap_pages, 32UL<<(20-PAGE_SHIFT));
> > +        xenheap_pages = min(xenheap_pages, 1UL<<(30-PAGE_SHIFT));
> > +    }
> >       do
> >       {
> 
> You seem to have forgotten to update the condition in the do/while:
> 
> xenheap_pages > 128 << (20 - PAGE_SHIFT);

Seems like I need to refactor this number into a constant somewhere.

Ian.

Re: [PATCH v2] xen: arm32: reduce default size of the xenheap

[Julien Grall]

Hi Ian,

On 05/02/2015 23:34, Ian Campbell wrote:
> On Thu, 2015-02-05 at 22:01 +0800, Julien Grall wrote:
>> If the user requests a xenheap of 0MB, we will use the default size,
>> right? It may be worth to explain this case.
>
> I think it's pretty generally understood that 0 generally means the
> default, unless otherwise specified.

IHMO, 0 means no xen heap.

In most of the options explained in the markdown file, they clearly 
explicit the behavior of 0 when necessary (see availmem).

Regards,

-- 
Julien Grall