Re: Strange interdependace between domains

[Nate Studer <nate.studer@dornerworks.com>]

On 2/14/2014 12:21 PM, Dario Faggioli wrote:
> On ven, 2014-02-14 at 12:02 +0000, Simon Martin wrote:
>> Thanks everyone and especially Ian! It was the hyperthreading that was
>> causing the problem.
>>
> Good to hear, and at the same time, sorry to hear that. :-)
> 
> I mean, I'm glad you nailed it, but at the same time, I'm sorry that the
> solution is to 'waste' a core! :-( I reiterate and restate here, without
> any problem doing so, the fact that Xen should be doing at least a bit
> better in these circumstances, if we want to properly address use cases
> like yours. However, there are limits on how far we can go, and hardware
> design is certainly among them!
> 
> All this to say that, it should be possible to get a bit more of
> isolation, by tweaking the proper Xen code path appropriately, but if
> the amount of interference that comes from two hypethreads sharing
> registers, pipeline stages, and whatever it is that they share, is
> enough for disturbing your workload, then, I'm afraid we never get much
> farther from the 'don't use hyperthread' solution! :-(

Which, as you say, unfortunately is the solution unless there is some way to
configure the hardware to eliminate this interference.  If it's any consolation,
the only multi-core ARINC653 implementations I know of have enacted these two
restrictions:
1.  # of cores enabled = # of memory controllers.
2.  Each enabled core must be configured to not share a memory controller,
cache, registers, etc...

It is practically an AMP system at that point, but without these restrictions
you can get some unpredictable behavior unless you have some specialized or
exotic hardware to make things more deterministic.

-- 
Nathan Studer
DornerWorks, Ltd.
Embedded Systems Engineering