Re: [PATCH v3 1/7] xen: vNUMA support for PV guests

[Dario Faggioli <dario.faggioli@citrix.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2785 bytes --]

On mar, 2013-11-26 at 16:59 -0500, Elena Ufimtseva wrote:
> Hello Jan and Dario.
> 
Hi,

> I have looked at what Jan asked and wanted to see if that can be resolved.
> 
> Jan is right, if the guest is running with Linux configured with
> maxcpus less than vcpus in VM config,
> there is a problem.
> 
Yep, there is indeed.

> On this boot stage where xen_numa_init is called xen_smp_prepare_cpus
> equals to vcpus in config;
> It only will be reduced to maxcpus (from kernel boot args) after
> xen_numa_init during xen_smp_prepare.
> 
> In xen_numa_init I have all values I need to make a decision in
> regards to initialize vnuma or not, or modify.
> 
> These are the numbers I have in xen_numa_init:
> num_possible_cpus() = hypervisor provided guest vcpus;
> setup_max_cpus = boot kernel param maxcpus;
> 
> When setup_max_cpus > num_possible_cpus, num_possible_cpus will be brought up;
> 
Wait. I think I see what you mean, but that's probably a different issue
from what (at least as far as I understood it) Jan is reporting.

So, in Linux you can specify NR_CPUS at compile time, right? Imagine
doing that and setting it to 1. Then, you use the resulting kernel for a
VM, and in the VM's config file you specify 4 vcpus and 2 vnodes.

Now, apart from what you say above, i.e., only num_possible_cpus()=1
will be brought up, that would also mean that, in xen_numa_init(), you
allocate an array only 1 element big for hosting vnuma data and pass it
to Xen, which will overflow it when trying to put info for 2 vnodes
there!

That's the thing we want to avoid...

> I can detect that setup_max_cpus < num_possible_cpus and do not init
> vnuma at all, and just do a fake node.
> I can also make sure that hypervisor is aware of it (by calling same
> subop with NULL, lets suppose).
> 
Right, but I'm not sure I'm getting how detecting that would relevant to
the issue above... What we'd need is something like
nr_vnodes<num_possible_cpus(), which means we need some mechanism to
retrieve the number of nodes _before_ allocating the arrays.

As per what to do, well, once we have a way to know te number of vnodes,
you just can allocate arrays of correct size and avoid this issue all
together. Of course, as we do not support guests with more nodes than
vcpus, at that time you can also check whether
nr_vnodes>num_possible_cpus() and, if yes, have xen_numa_init() take
some error path, but that is a different thing.

Does this make sense?

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel