Re: Strange interdependace between domains

[Simon Martin <furryfuttock@gmail.com>]

Hello Dario,

> All this to say that, it should be possible to get a bit more of
> isolation, by tweaking the proper Xen code path appropriately, but if
> the amount of interference that comes from two hypethreads sharing
> registers, pipeline stages, and whatever it is that they share, is
> enough for disturbing your workload, then, I'm afraid we never get much
> farther from the 'don't use hyperthread' solution! :-(

Hyperthreading is just a way to improve CPU resource utilization. Even
if you are doing a CPU intensive operation, a lot of the processor
circuits are actually idle, so adding 2 pipelines to feed one
processor is a good way to improve total throughput, but it does have
it have it's caveats. I totally forgot this.

Given the way that this works there isn't much that Xen can do. It is
a physical restriction.

The only thing I can think of would be to add an option to only run
hardware interaction on a sub-set of the available hyperthreads, but
that would require hacking the Dom0 kernel to lock device drivers onto
a given set of CPU and might end up worst than when it started.

Another way might be to have overlapping CPU pools. That way I have
Dom0 running on only one core and then I can run DomUs spread over the
available cores. AFAIU it is the hardware interaction that is causing
the interdependency.

> Anyways, with respect to the first part of this reasoning, would you
> mind (when you've got the time of course) one more test? If no, I'd say,
> configure the system as I was suggesting in my first reply, i.e., using
> also core #2 (or, in general, all the cores). Also, make sure you add
> this parameter, to the Xen boot command line:

>  sched_smt_power_savings=1

> (some background here:
> http://lists.xen.org/archives/html/xen-devel/2009-03/msg01335.html)

> And then run the bench with disk activity on.

OK. This is my current configuration:

Dom0     PCPU 0,1,2   no pinning
win7x64  PCPU   1,2   pinned
pv499    PCPU       3 pinned

And I get the same interdependence.

root@smartin-xen:~# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  1253     3     r-----      37.1
win7x64                                      4  2046     2     ------     106.8
pv499                                        5   128     1     r-----      66.8
root@smartin-xen:~# xl vcpu-list 
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0    1   -b-      21.2  all
Domain-0                             0     1    2   r--       8.8  all
Domain-0                             0     2    0   -b-       8.2  all
win7x64                              4     0    1   -b-      59.8  1
win7x64                              4     1    2   -b-      49.0  2
pv499                                5     0    3   r--      70.2  3
root@smartin-xen:~# xl cpupool-list -c
Name               CPU list
Pool-0             0,1,2
pv499              3

I have gone back to my working settings.


-- 
Best regards,
 Simon                            mailto:furryfuttock@gmail.com