Re: Strange interdependace between domains

[Simon Martin <furryfuttock@gmail.com>]

Thanks for all the replies guys.

With respect your comments/queries I'll put them all in here.

>> 1.- Hardware: Intel i3, 4GB RAM, 64GB SSD.
Andrew> Can you be more specific - this covers 4 generations of Intel CPUs.

This is a 4th gen (Haswell) processor.

>> # xl cpupool-list
>> Name               CPUs   Sched     Active   Domain count
>> Pool-0               3    credit       y          2
>> pv499                1  arinc653       y          1
Dario> Ok, I think I figured this out from the other information, but it would
Dario> be useful to know what pcpus are assigned to what cpupool. I think it's
Dario> `xl cpupool-list -c'.

Pool-0: 0,1,2
Dom0: 3

Don> How many instruction per second a thread gets does depend on the
Don> "idleness" of other threads (no longer just the hyperThread's
Don> parther).

This    seems    a    bit    strange   to   me. In my case I have time
critical  PV  running  by  itself  in a CPU pool. So Xen should not be
scheduling it, so I can't see how this Hypervisor thread would be affected.

>> 6.- All VCPUs are pinned:
>> 
Dario> Right, although, if you use cpupools, and if I've understood what you're
Dario> up to, you really should not require pinning. I mean, the isolation
Dario> between the RT-ish domain and the rest of the world should be already in
Dario> place thanks to cpupools.

This  is what I thought, however when running looking at the vcpu-list
I  CPU  affinity  was "all" until I starting pinning. As I wasn't sure
whether  that  was  "all  inside this cpu pool" or "all" I felt it was
safer to do it explicitly.

Dario> So, if you ask me, you're restricting too much things in
Dario> pool-0, where dom0 and the Windows VM runs. In fact, is there a
Dario> specific reason why you need all their vcpus to be statically
Dario> pinned each one to only one pcpu? If not, I'd leave them a
Dario> little bit more of freedom.

I agree with you here, however when I don't pin CPU affinity is "all".
Is this "all in the CPU pool"? I couldn't find that info.

Dario> What I'd try is:
Dario>  1. all dom0 and win7 vcpus free, so no pinning in pool0.
Dario>  2. pinning as follows:
Dario>      * all vcpus of win7 --> pcpus 1,2
Dario>      * all vcpus of dom0 --> no pinning
Dario>    this way, what you get is the following: win7 could suffer sometimes,
Dario>    if all its 3 vcpus gets busy, but that, I think is acceptable, at
Dario>    least up to a certain extent, is that the case?
Dario>    At the same time, you
Dario>    are making sure dom0 always has a chance to run, as pcpu#0 would be
Dario>    his exclusive playground, in case someone, including your pv499
Dario>    domain, needs its services.

This  is  what  I  had when I started :-). Thanks for the confirmation
that I was doing it right. However if the hyperthreading is the issue,
then I will only have 2 PCPU available, and I will assign them both to
dom0 and win7.

Dario> Right. Are you familiar with tracing what happens inside Xen
Dario> with xentrace and, perhaps, xenalyze? It takes a bit of time to
Dario> get used to it but, once you dominate it, it is a good mean for
Dario> getting out really useful info!

Dario> There is a blog post about that here:
Dario> http://blog.xen.org/index.php/2012/09/27/tracing-with-xentrace-and-xenalyze/
Dario> and it should have most of the info, or the links to where to
Dario> find them.

Thanks for this. If this problem is more than the hyperthreading then
I will definitely use it. Also looks like it might be useful when I
start looking at the jitter on the singleshot timer (which should be
in a couple of weeks).

Andrew> How are you measuring time in pv499?

In  two  ways: counting the periodic interrupts (125 microsecond) and
from the monotonic_clock. They both agree on the time.

Andrew> What is your Cstates and Pstates looking like?

No  idea.  If the hyperthreading suggestion doesn't work out I'll look
at this.

Andrew> If you can, try disabling turbo?

If the hyperthreading suggestion doesn't work out I'll look at this.
-- 
Best regards,
 Simon                            mailto:furryfuttock@gmail.com