Re: PCI passthrough issue

[Jean Baptiste Favre <xen-devel@jbfavre.org>]

Le 01/02/2011 15:18, Ian Campbell a écrit :
> On Tue, 2011-02-01 at 14:12 +0000, Jean Baptiste Favre wrote:
>
> > Le 01/02/2011 14:20, Ian Campbell a écrit :
>
> >>>> If you restrict dom0 to >256MB but <512MB (using dom0_mem= on
> >> hypervisor
> >>>> command line) does the NIC work correctly in non-passedthrough form?
> >>> My Xen hypervisor commandline is as follow:
> >>> placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all
> >>> guest_loglvl=all com1=115200,8n1 console=com1
> >>>
> >>> Everything works great without passthrough, but my dom0 is 64bits
> which
> >>> may explain that (I do have this strange behaviour only with 32bits
> >>> kernels).
> >>
> >> I don't suppose you could try installing a 32 bit OS in dom0? (e.g.
> as a
> >> skanky hack you might fit something into your existing swap
> >> partition ;-))
> > I could try, but that would means breaking my test platform. Let's say I
> > prefer complete other tests before :)
>
> Sure, no problem.
>
> >>> I did not tried changing dom0_mem param.
> >>>
> >>>> Similarly does the kernel running native with mem= cause the failure?
> >>> Not sure I understand what you mean here.
> >>
> >> I meant to run the system as a native (non-Xen) system and use the
> >> kernels mem= command line parameter to restrict the system to the
> >> problematic amounts of memory. Really just trying to verify if
> something
> >> is up specifically due to Xen or not. Probably needs a 32 bit
> >> user/kernel to be a useful experiment.
> > Ah ok. But then, running 32bits kernel with 64 bits OS will work ? (I
> > didn't know 64bits kernel worked with 32bits OS before Konrad told me)
>
> 32 user on 64 kernel works, 64 user on 32 kernel does not so this will
> tie in with the 32 bit test above too.
>
> >> What do "ifconfig" or "ethtool -S" show for the device after some
> >> failures. Do any of the numbers increment inline with the dropped
> >> frames?
> >>
> >> Perhaps interestingly 85 bytes, plus the ICMP, IP and Ethernet header
> >> sizes is something around 128 bytes (depending on options etc). This is
> >> pure numerology but I notice that sky2 has a copybreak parameter
> >> ("Receive copy threshold") which defaults to 128. I think it would be
> >> worth trying 64 and 256.
> > That's exactly the problem I faced with 256mb memory for my domU.
> > Outgoing packets reached my gateway (tcpdump saw them on it, as well as
> > replies) but incoming packets greater than 128bits were blocked
> > somewhere between Xen hypervisor and domU user space (where I was
> > listening traffic with tcpdump).
> >
> > I didn't notice the copybreak from sky2. Where can I change it ? It
> > could be exactly what we're looking for from the beginning, couldn't
> it ?
> > How can I change it ?
>
> Assuming the driver is modular:
>         "modprobe sky2 copybreak=<N>"
>
> Depending on your distro there will be somewhere in /etc you can add
> this. e.g. on Debian you can create a file in /etc/modprobe.d/
> containing "option sky copybreak=<N>" other distros
> use /etc/modprobe.conf etc.
OK I see but it doesn't seems to have any effect.
I tried "option sky copybreak=0" to get all packet copied with no change.

But I have to say that I'm a bit confused: as I run a PV domU, kernel
and initrd are provided by dom0.
So basically, I had no module related binaries installed. After
installation, I tried to remove module and reload it with different
configuration without changes.
Is there any way to provide this sort of option in kernel commandline so
that it 'll be taken into account even in initrd ?

> >> Are you able to see any traffic with tcpdump, either in guest and/or
> >> from another host (may require switch configuration to allow it to see
> >> the traffic). e.g. do you see the ICMP Echo Request but not the Echo
> >> Response or do you see nothing at all etc.
> > tcpdump tests have been done from my gateway and from domU.
> > On the GW: can see all incoming packets (whatever could be the size) and
> > send all replies.
> > On the DomU: can see all outgoing packets but only incoming ones shorter
> > than 128bytes (global size, means "ping -s85" and less)
>
> OK, so it is the sky2 receive path which is at fault, that's very useful
> info. It corresponds with the copybreak theory too.
And that makes me happy, because I began to desperate :)

JB