Re: PCI passthrough issue

[Jean Baptiste Favre <xen-devel@jbfavre.org>]

Le 12/01/2011 18:26, Konrad Rzeszutek Wilk a écrit :
> On Wed, Jan 12, 2011 at 05:56:45PM +0100, Jean Baptiste Favre wrote:
>> Le 12/01/2011 17:36, Konrad Rzeszutek Wilk a écrit :
>>> On Wed, Jan 12, 2011 at 04:53:51PM +0100, Jean Baptiste Favre wrote:
>>>> Hello Konrad,
>>>>
>>>> Le 12/01/2011 16:43, Konrad Rzeszutek Wilk a écrit :
>>>>> On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote:
>>>>>> Hello,
>>>>>> I'm facing a strange issue with network card PCI passthrough on my
>>>>>> openwrt test domU.
>>>>>>
>>>>>> - With network PCI passthrough, DNS lookup failed for some domains
>>>>>> (exemple, google.com) but not for other (free.fr my ISP, or my domain
>>>>>> jbfavre.org). I can ping an IP address without any problem.
>>>>>
>>>>> Do you have "both" (so PCI passthrough and the Xen network driver)
>>>>> in the guest? If so, have you tried eliminating the xen network driver
>>>>> to see if it is just a routing issue?
>>>> Have not tried to eliminate xen network driver. Think I have both drivers.
>>>>
>>>> My kernel .config looks like:
>>>> $ grep XEN build_dir/linux-x86_xen_domu/linux-2.6.37/.config
>>>> CONFIG_XEN=y
>>>> # CONFIG_XEN_PRIVILEGED_GUEST is not set
>>>> CONFIG_XEN_PVHVM=y
>>>> CONFIG_XEN_MAX_DOMAIN_MEMORY=128
>>>> CONFIG_XEN_SAVE_RESTORE=y
>>>> CONFIG_PCI_XEN=y
>>>> CONFIG_XEN_PCIDEV_FRONTEND=y
>>>> CONFIG_XEN_BLKDEV_FRONTEND=y
>>>> CONFIG_XEN_NETDEV_FRONTEND=y
>>>> CONFIG_HVC_XEN=y
>>>> CONFIG_XEN_BALLOON=y
>>>> CONFIG_XEN_SCRUB_PAGES=y
>>>> CONFIG_XEN_DEV_EVTCHN=y
>>>> CONFIG_XENFS=y
>>>> CONFIG_XEN_COMPAT_XENFS=y
>>>> CONFIG_XEN_SYS_HYPERVISOR=y
>>>> # CONFIG_XEN_PLATFORM_PCI is not set
>>>> CONFIG_SWIOTLB_XEN=y
>>>>
>>>> So, I should remove CONFIG_XEN_NETDEV_FRONTEND ?
>>>
>>> No. You can just do 'ifconfig <x> down' whatever your Xen netfront
>>> NIC is. .. but.
>>>>
>>>>> What does your routing table look like? Your IP table?
>>>> My routing table is pretty clean, nothing strange here
>>>> # route -n
>>>> Kernel IP routing table
>>>> Destination     Gateway         Genmask         Flags Metric Ref    Use
>>>> Iface
>>>> 10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0
>>>> br-wan
>>>> 0.0.0.0         10.0.0.1        0.0.0.0         UG    0      0        0
>>>> br-wan
>>>
>>> Ok, then the idea that the Xen networking driver and the PCI passthrough send
>>> packets over is not the way..
>>>
>>> So ignore about the Xen networking part.
>>
>> OK.
>>
>>>>
>>>>>> - Starting domU as a "normal" (ie without PCI passthrough), no problem.
>>>>>>
>>>>>>
>>>>>> As far as I can say, domU is not the root cause. I really think this is
>>>>>> related to PCI passsthrough. This seems to be related to packet length.
>>>>>
>>>>> Then that would imply the MTU is not set right. 
>>>> Already checked it: 1500 :)
>>> Ok.
>>>
>>> Next thing, did you try to disable the rx/tx checksumming?
>>
>> No. Not sure I even know how to do it, but will have a look on that.
>>
>>> If you connect the Ethernet cable for this 'br-wan' device to another machine
>>> (so you could set it up as bridge and just let it pass through packets and sniff
>>> the data) what do the packets look like?
>>
>> I can see packets on my gateway (which acts as DNS as well). They looks
>> good. I can see DNS answers as well leaving the gateway, but not
>> reaching my domU. Have only basic switches between my gateway and my domU.
> 
> Wait a minute. How are your domU's connected? You did turn on
> forwarding in your guest with the PCI passthrough right?

My OpenWRT domU with PCI passthrough is directly connected to my LAN
through network card

> How is your domain (with the PCI passthrought) connected to the
> other domains?

For the moment, other domU are not connected in any way to my OpenWRT
domU. I want first to make sur PCI passthrough is working before trying
more complex architecture.

My OpenWRT domU network setup is:

dom0           domU            LAN

vif ----- eth0      eth1 ----- SW ----- GW ----- NET
            |__ br __|

All tests are done from OpenWRT domU for now (but I guess this won't
work either from dom0).

I also tried following setup:
domU            LAN

    eth1 ----- SW ----- GW ----- NET
 br __|

That means no vif from dom0. But with PCI passthrough, I should reach
network directly, right ?

Finally, I think that my setup shall not be responsible for the fact
that ping works, small DNS answers are received but not bigger ones
(limit seems to be around 100 bytes).

>>> What happens if the PCI passthrough device is not under the ownership of a bridge?
>>> What then? You wouldn't have any bridge firewall code in?
>>
>> Removed bridge + configure eth: no change.
>> Checked ebtables/iptables (all tables: net filter and mangle) rules:
>> empty, Policy to ACCEPT.
>> Tried to enable/disable ip_forwarding: no change
>> Also disabled IPV6 support (I saw some DNS answers as AAAA but no IPv6
>> available at home for now): no change
>>
>> Regards,
>> JB