Re: [PATCH v2 0/7] vNUMA introduction.

[Elena Ufimtseva <ufimtseva@gmail.com>]

On Thu, Nov 14, 2013 at 5:03 AM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> Hi again, Elena,

Hello Dario )
>
> And thanks for the good work! :-)
>
> On mer, 2013-11-13 at 22:25 -0500, Elena Ufimtseva wrote:
>> vNUMA introduction
>>
>> [...]
>>
>> This patchset can be pulled from https://git.gitorious.org/xenvnuma/xenvnuma.git
>> Linux patchset https://git.gitorious.org/xenvnuma/linuxvnuma.git
>>
> AhA! When replying to the Linux series, I said the linuxvnuma.git repo
> seemed empty, but trying again now it's cloning something... I guess you
> either fixed this or it was just me having/bad timing. :-)

Yes, gitorious is not that reliable when it comes to bigger repository.
>
>> Examples of booting vNUMA enabled PV Linux guest on real NUMA machine:
>>
>> 1. Automatic vNUMA placement on real NUMA machine:
>>
>> VM config:
>>
>> memory = 16384
>> vcpus = 4
>> name = "rcbig"
>> vnodes = 4
>> vnumamem = [10,10]
>> vnuma_distance = [10, 30, 10, 30]
>> vcpu_to_vnode = [0, 0, 1, 1]
>>
>> [..]
>>
>> Comments:
>> None of the configuration options are correct so default values were used.
>>
> And (talking without having looked at the patches yet), you do print a
> warning when this happens, right? :-)
>
>> Notes:
>> *   to enable vNUMA in linux kernel the corresponding patch set should be
>>     applied;
>> *   automatic numa balancing featurue seem to be fixed in linux kernel:
>>     https://lkml.org/lkml/2013/7/31/647
>>
> Mmm... I'm quite curious about this, since we talked extensively about
> it in Edinburgh. Does this mean you're not having issues with the NUMA
> hinting page fault any longer? Even without doing anything, either in
> Xen or Linux?

Correct! :) I have run multiple tests, including kernel compilation.
I have reverted all numa balancing related code I introduced and checked again.
And now with automatic numa balancing turened on there is no such issue.
The only thing there is a potential for oops on linux kernel side when
migrating huge pages
(set_pmd_at is absent in pv_mmu_ops, but I will see if I can catch it
wirth testing).

>
> So what was that? The URL above is just someone reporting a quite
> general 'memory corruption issue', and nothing about what the cause was
> and whether and how it has been fixed? But even more important, was that
> it that was causing the problem you were seeing?

My apologies, correct link is as follows
https://lkml.org/lkml/2013/10/31/133
Thats Ingo Molnar's work. I did not look precisely into the code yet
and will do it today, but from the first glance
things has changed quite a bit.

>
>> TODO:
>> *   This version limits vdistance config option to only two values - same node
>>     distance and other node distance; This prevents oopses on latest (3.13-rc1)
>>     linux kernel with non-symmetric distance;
>>
> Ok, that's fine for now. We'll work on allowing the syntax we agreed
> with IanJ during last round of review (and yes, with "We", I mean
> "I" :-D).
>
>> *   cpu siblings for Linux machine and xen cpu trap should be detected and
>>     warning should be given; Add cpuid check if set in VM config;
>> *   benchmarking;
>>
> That's a big one! I think I'll have something ready at least for
> facilitating it soon.

I would like to know how you do this :)
>
> Thanks again and Regards,
> Dario
>
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>



-- 
Elena