Question about hyper-threading of domains

[Jacob Gorm Hansen <jacobg@diku.dk>]

hi,

is it possible, and will it make sense, to allow an SMP-domain access to 
both SMT/Hyper threads of a CPU?

My performance measurements of the small address spaces implementation 
(SAS) seem to indicate that having the driver domain in a separate hyper 
thread is still a little faster than SAS for a non-SMP/SMT domU.  I 
guess that means that that SMT IPIs are about the same cost as ring 
switches.

So quite likely SAS is only interesting if:

1) You don't have SMT (luckily, I have a bunch of machines like that 
back home ;-)). Do the recent Opterons and Itaniums have SMT btw?

2) You want to run SMT in the domUs.

3) You give up driver isolation and manage to get dom0 running in ring0 
(to remove the cost of ring switches). Some comments in the Xen code 
indicate that this will be hard to do, so it may not be worth it.

I previously stated that using small address spaces means giving up 
driver isolation, but I think that with the right use of segments that 
should not be necessary, so there is no trade-off with regards to isolation.

My patch to Xen and XenLinux is fairly small, though a few things 
(mainly the PERDOMAIN mapping and thus use of segments in domains) are 
not handled correctly at this point. Also, I do not use segments to keep 
domUs out of dom0, which I will at some point.

Basically, I have added per-domain pgd lower and upper bounds, and I 
have added an extra pgd slot for the linear page tables of dom0. The 
linear_* macros now take an exec_domain* as argument, and I have a 
pgd-version check in __context-switch() that potentially updates the 
cached entries at the top of the domUs pgd, and does nothing (i.e. does 
not write cr3) otherwise.  If needed, I flush the TLB before 
pdg-updates,  and when unpinning a pgd I clear the cached area before 
put_page() and friends.

Most of the changes are of a nature that could be made applicable to 
other architectures as well. I think they could be activated at run-time 
with little or no overhead when not in use.

I would be happy to share my changes with anyone interested.

Jacob