Re: X58 Virtualization w/ Linux

Steve Novakov
2016-06-15T16:01:48Z

Hello Paolo, thank you for joining in,

Yeah absolutely, it freezes no matter what I do. I get all sorts of
interesting error messages during boot. Dmesg goes to like [64.4XXXX] or
higher and spits out random kernel functions, addresses, etc. then
freezes. I've tried

intel_iommu=on
intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1

all sorts of combinations. I've verified that all of the relevant
modules load (vfio* kvm kvm_intel pci_stub) through dmesg. That that
"DMAR-IR" message is suspicious though, no? That there is some setting
in the kernel that disables remapping by default for my BIOS. I've seen
posts with working vt-d even with an i7-950. I wonder if they have the
C3 stepping of the northbridge. Does that sound like it might be an issue?

I can't believe that there is not some way to make this work, as it
works flawlessly with ESXi (which sits on top of a linux kernel, though
it's vmware's proprietary fork I think). If my system boots in "Plug and
play" mode, the BIOS configuration for non critical devices shouldn't
matter, right?

Steve Novakov
B.A.Sc. Engineering Physics
PhD Student - Physics
University of Michigan
On 6/15/2016 2:59 AM, Paolo Bonzini wrote:
>
> On 15/06/2016 07:04, Steve Novakov wrote:
>> So I swapped over to ubuntu 16.04 server and I just saw this line in dmesg:
>>
>> [0.100541] DMAR-IR: This system BIOS has enabled interrupt remapping on
>> a chipset that contains an erratum making that feature unstable. To
>> maintain system stability interrupt remapping is being disabled.please
>> contact your BIOS vendor for an update.
>>
>> This line was actually present while using arch as well I just never
>> took the time to read past the first line (which just says enabled
>> interrupt remapping). This seems to be an issue with northbridge
>> stepping B3, and was apparently fixed in stepping C2 ( I have B3 : ( ).
>>
>> I wonder if it's even possible to get this to work with my motherboard
>> now....**though ESXi can manage just fine...**
>>
>> Maybe there's some way to trick linux into just trying to do it. I mean,
>> it seems to pass devices to pci-stub just fine.
> Passing devices to pci-stub doesn't require the IOMMU, it's just
> swapping the driver that binds to the device. I think this is why with
> VT-d disabled you cannot assign the device to vfio-pci.
>
> Does it still freeze with Ubuntu 16.04 (VT-d enabled,
> vfio_iommu_type1.allow_unsafe_interrupts=1)?
>
> Thanks,
>
> Paolo