Re: [Qemu-devel] PCI passthrough of 40G ethernet interface (Openstack/KVM)

[Bandan Das <bsd@redhat.com>]

jacob jacob <opstkusr@gmail.com> writes:

> On Mon, Mar 16, 2015 at 2:12 PM, Bandan Das <bsd@redhat.com> wrote:
>> jacob jacob <opstkusr@gmail.com> writes:
>>
>>> I also see the following in dmesg in the VM.
>>>
>>> [    0.095758] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
>>> [    0.096006] acpi PNP0A03:00: ACPI _OSC support notification failed,
>>> disabling PCIe ASPM
>>> [    0.096915] acpi PNP0A03:00: Unable to request _OSC control (_OSC
>>> support mask: 0x08)
>> IIRC, For OSC control, after BIOS is done with (whatever initialization
>> it needs to do), it clears a bit so that the OS can take over. This message,
>> you are getting is a sign of a bug in the BIOS (usually). But I don't
>> know if this is related to your problem. Does "dmesg | grep -e DMAR -e IOMMU"
>> give anything useful ?
>
> Do not see anything useful in the output..

Ok, Thanks. Can you please post the output as well ?

>>> [    0.097072] acpi PNP0A03:00: fail to add MMCONFIG information,
>>> can't access extended PCI configuration space under this bridge.
>>>
>>> Does this indicate any issue related to PCI passthrough?
>>>
>>> Would really appreciate any input on how to bebug this further.
>>
>> Did you get a chance to try a newer kernel ?
> Currently am using 3.18.7-200.fc21.x86_64 which is pretty recent.
> Are you suggesting trying the newer kernel just on the host? (or VM too?)
Both preferably to 3.19. But it's just a wild guess. I saw i40e related fixes,
particularly "i40e: fix un-necessary Tx hangs" in 3.19-rc5. This is not exactly
what you are seeing but I was still wondering if it could help.

Meanwhile, I am trying to get hold of a card myself to try and reproduce
it at my end.

Thanks,
Bandan

>>> On Fri, Mar 13, 2015 at 10:08 AM, jacob jacob <opstkusr@gmail.com> wrote:
>>>>> So, it could be the i40e driver then ? Because IIUC, VFs use a separate
>>>>> driver. Just to rule out the possibility that there might be some driver fixes that
>>>>> could help with this, it might be a good idea to try a 3.19 or later upstream
>>>>> kernel.
>>>>>
>>>>
>>>> I tried with the latest DPDK release too (dpdk-1.8.0) and see the same issue.
>>>> As mentioned earlier, i do not see any issues at all when running
>>>> tests using either i40e or dpdk on the host itself.
>>>> This is the reason why i am suspecting if it is anything to do with KVM/libvirt.
>>>> Both with regular PCI passthrough and VF passthrough i see issues. It
>>>> is always pointing to some issue with packet transmission. Receive
>>>> seems to work ok.
>>>>
>>>>
>>>> On Thu, Mar 12, 2015 at 8:02 PM, Bandan Das <bsd@redhat.com> wrote:
>>>>> jacob jacob <opstkusr@gmail.com> writes:
>>>>>
>>>>>> On Thu, Mar 12, 2015 at 3:07 PM, Bandan Das <bsd@redhat.com> wrote:
>>>>>>> jacob jacob <opstkusr@gmail.com> writes:
>>>>>>>
>>>>>>>>  Hi,
>>>>>>>>
>>>>>>>>  Seeing failures when trying to do PCI passthrough of Intel XL710 40G
>>>>>>>> interface to KVM vm.
>>>>>>>>      0a:00.1 Ethernet controller: Intel Corporation Ethernet
>>>>>>>> Controller XL710 for 40GbE QSFP+ (rev 01)
>>>>>>>
>>>>>>> You are assigning the PF right ? Does assigning VFs work or it's
>>>>>>> the same behavior ?
>>>>>>
>>>>>> Yes.Assigning VFs worked ok.But this had other issues while bringing down VMs.
>>>>>> Interested in finding out if PCI passthrough of 40G intel XL710
>>>>>> interface is qualified in some specific kernel/kvm release.
>>>>>
>>>>> So, it could be the i40e driver then ? Because IIUC, VFs use a separate
>>>>> driver. Just to rule out the possibility that there might be some driver fixes that
>>>>> could help with this, it might be a good idea to try a 3.19 or later upstream
>>>>> kernel.
>>>>>
>>>>>>>> From dmesg on host:
>>>>>>>>
>>>>>>>>> [80326.559674] kvm: zapping shadow pages for mmio generation wraparound
>>>>>>>>> [80327.271191] kvm [175994]: vcpu0 unhandled rdmsr: 0x1c9
>>>>>>>>> [80327.271689] kvm [175994]: vcpu0 unhandled rdmsr: 0x1a6
>>>>>>>>> [80327.272201] kvm [175994]: vcpu0 unhandled rdmsr: 0x1a7
>>>>>>>>> [80327.272681] kvm [175994]: vcpu0 unhandled rdmsr: 0x3f6
>>>>>>>>> [80327.376186] kvm [175994]: vcpu0 unhandled rdmsr: 0x606
>>>>>>>
>>>>>>> These are harmless and are related to unimplemented PMU msrs,
>>>>>>> not VFIO.
>>>>>>>
>>>>>>> Bandan
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html