Re: PV-vNUMA issue: topology is misinterpreted by the guest

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 07/27/2015 10:43 AM, Juergen Gross wrote:
> On 07/27/2015 04:34 PM, Boris Ostrovsky wrote:
>> On 07/27/2015 10:09 AM, Dario Faggioli wrote:
>>> On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote:
>>>> On 07/24/2015 05:58 PM, Dario Faggioli wrote:
>>>>> So, just to check if I'm understanding is correct: you'd like to 
>>>>> add an
>>>>> abstraction layer, in Linux, like in generic (or, perhaps, 
>>>>> scheduling)
>>>>> code, to hide the direct interaction with CPUID.
>>>>> Such layer, on baremetal, would just read CPUID while, on PV-ops, 
>>>>> it'd
>>>>> check with Xen/match vNUMA/whatever... Is this that you are saying?
>>>> Sort of, yes.
>>>>
>>>> I just wouldn't add it, as it is already existing (more or less). It
>>>> can deal right now with AMD and Intel, we would "just" have to add 
>>>> Xen.
>>>>
>>> So, having gone through the rest of the thread (so far), and having
>>> given a fair amount o thinking to this, I really think that something
>>> like this would be a good thing to have in Linux.
>>>
>>> Of course, it's not that my opinion on where should be in Linux counts
>>> that much! :-D   Nevertheless, I wanted to make it clear that, while
>>> skeptic at the beginning, I now think this is (part of) the way to go,
>>> as I said and explained in my reply to George.
>>
>> And I continue to believe that kernel solution does not address the
>> userland problem which is no less important than making kernel do proper
>> scheduling decisions (and I suspect when this patch goes for review
>> that's what the scheduling people are going to say).
>>
>> Remember the original problem that started this thread was that kernel
>> complained that topology didn't make sense and it turned off all
>> topology-related decisions. Which means that kernel already has a
>> solution for weird topology. Some enumeration doesn't trigger this
>> warning, but we can come up with one that does. Or we can indeed have a
>> patch in kernel that will, possibly silently, fail topology_sane() when
>> virtualized and not pinned.
>
> How would you come up with a topology the kernel is complaining about
> and user mode scheduling will use for sane decisions ?

We need to understand first why Dario's box is apparently the only one 
resulting in a warning and probably then emulate that enumeration.

And again, if that is not possible then just make topology_sane() fail.

>
>> (This is what I assume kernel does when topology_sane() fails. And if it
>> doesn't, that's a bug IMO)
>>
>> The licensing problem that Juergen described can be solved by pining
>> vcpus and exposing HT bit. Besides,  creating a guest with 24 VPCUs and
>
> Hmm, yes. This way you sacrifice most of the virtualization advantages.
>
>> hoping that 16-core licensing will work I think is pushing it a bit when
>> you know that VCPUs will jump around cores (i.e. "on average" you are
>> running on more than 16 cores -- multi-threaded or not -- which arguably
>> is what licensing is trying to prevent)
>
> On a machine with only 16 cores running on more than 16 cores? I have
> some problems to believe this. The point was: if the license is happy on
> bare metal it should be so when running on the same hardware as a guest.

Ok, that's not how I should have described it. I meant that IMO asking 
for 24 VCPUs is somewhat akin to oversubscribing since you kind of know 
that you dont' have 24 PCPUs, you are just trying to fool the kernel 
into thinking that threads are cores.

-boris