Re: Question about hyper-threading of domains

[Lars Roland <lroland@gmail.com>]

On 4/29/05, Jacob Gorm Hansen <jacobg@diku.dk> wrote:
> hi,
> 
> is it possible, and will it make sense, to allow an SMP-domain access to
> both SMT/Hyper threads of a CPU?
> 
> My performance measurements of the small address spaces implementation
> (SAS) seem to indicate that having the driver domain in a separate hyper
> thread is still a little faster than SAS for a non-SMP/SMT domU.  I
> guess that means that that SMT IPIs are about the same cost as ring
> switches.

Seans fair enough - Out of curiosity have you published these
measurements anywere ?

> 
> So quite likely SAS is only interesting if:
> 
> 1) You don't have SMT (luckily, I have a bunch of machines like that
> back home ;-)). Do the recent Opterons and Itaniums have SMT btw?

No neither the latest Opterons nor the Itaniums have explicit SMT.

> 
> 2) You want to run SMT in the domUs.
> 
> 3) You give up driver isolation and manage to get dom0 running in ring0
> (to remove the cost of ring switches). Some comments in the Xen code
> indicate that this will be hard to do, so it may not be worth it.
> 
> I previously stated that using small address spaces means giving up
> driver isolation, but I think that with the right use of segments that
> should not be necessary, so there is no trade-off with regards to isolation.

Could you explain a little further how excalty you plan to avoid this
- I have been digging into this myslef and It is not obvious to me how
this can be achived,

> 
> My patch to Xen and XenLinux is fairly small, though a few things
> (mainly the PERDOMAIN mapping and thus use of segments in domains) are
> not handled correctly at this point. Also, I do not use segments to keep
> domUs out of dom0, which I will at some point.
> 
> Basically, I have added per-domain pgd lower and upper bounds, and I
> have added an extra pgd slot for the linear page tables of dom0. The
> linear_* macros now take an exec_domain* as argument, and I have a
> pgd-version check in __context-switch() that potentially updates the
> cached entries at the top of the domUs pgd, and does nothing (i.e. does
> not write cr3) otherwise.  If needed, I flush the TLB before
> pdg-updates,  and when unpinning a pgd I clear the cached area before
> put_page() and friends.
> 
> Most of the changes are of a nature that could be made applicable to
> other architectures as well. I think they could be activated at run-time
> with little or no overhead when not in use.
> 
> I would be happy to share my changes with anyone interested.

Yes please - I have the time (and hardware) to check this.


Regards.

Lars Roland