Re: X58 Virtualization w/ Linux

Yang Zhang
2016-06-16T01:19:33Z

On 2016/6/15 23:40, Steve Novakov wrote:
> Hello Paolo, thank you for joining in,
>
> Yeah absolutely, it freezes no matter what I do. I get all sorts of
> interesting error messages during boot. Dmesg goes to like [64.4XXXX] or
> higher and spits out random kernel functions, addresses, etc. then
> freezes. I've tried
>
> intel_iommu=on
> intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1
>
> all sorts of combinations. I've verified that all of the relevant
> modules load (vfio* kvm kvm_intel pci_stub) through dmesg. That that
> "DMAR-IR" message is suspicious though, no? That there is some setting
> in the kernel that disables remapping by default for my BIOS. I've seen
> posts with working vt-d even with an i7-950. I wonder if they have the
> C3 stepping of the northbridge. Does that sound like it might be an issue?
>
> I can't believe that there is not some way to make this work, as it
> works flawlessly with ESXi (which sits on top of a linux kernel, though
> it's vmware's proprietary fork I think). If my system boots in "Plug and
> play" mode, the BIOS configuration for non critical devices shouldn't
> matter, right?

It seems there are something wrong with vfio. Have you tried with legacy
device pass through? It should be disabled on new distribution(like
ubuntu 16.04) but RHEL6 may still support it.

>
> Steve Novakov
> B.A.Sc. Engineering Physics
> PhD Student - Physics
> University of Michigan
> On 6/15/2016 2:59 AM, Paolo Bonzini wrote:
>>
>> On 15/06/2016 07:04, Steve Novakov wrote:
>>> So I swapped over to ubuntu 16.04 server and I just saw this line in
>>> dmesg:
>>>
>>> [0.100541] DMAR-IR: This system BIOS has enabled interrupt remapping on
>>> a chipset that contains an erratum making that feature unstable. To
>>> maintain system stability interrupt remapping is being disabled.please
>>> contact your BIOS vendor for an update.
>>>
>>> This line was actually present while using arch as well I just never
>>> took the time to read past the first line (which just says enabled
>>> interrupt remapping). This seems to be an issue with northbridge
>>> stepping B3, and was apparently fixed in stepping C2 ( I have B3 : ( ).
>>>
>>> I wonder if it's even possible to get this to work with my motherboard
>>> now....**though ESXi can manage just fine...**
>>>
>>> Maybe there's some way to trick linux into just trying to do it. I mean,
>>> it seems to pass devices to pci-stub just fine.
>> Passing devices to pci-stub doesn't require the IOMMU, it's just
>> swapping the driver that binds to the device. I think this is why with
>> VT-d disabled you cannot assign the device to vfio-pci.
>>
>> Does it still freeze with Ubuntu 16.04 (VT-d enabled,
>> vfio_iommu_type1.allow_unsafe_interrupts=1)?
>>
>> Thanks,
>>
>> Paolo
>

--
best regards
yang