From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jean Baptiste Favre Subject: Re: PCI passthrough issue Date: Tue, 01 Feb 2011 16:38:01 +0100 Message-ID: <4D4828D9.6090601@jbfavre.org> References: <4D2E28C5.30203@jbfavre.org> <4D2EE1DE.5070006@jbfavre.org> <4D2F5009.2090701@jbfavre.org> <20110113201922.GA20494@dumpdata.com> <4D2F6431.8030606@jbfavre.org> <20110114145350.GB7371@dumpdata.com> <4D30DC5A.9080303@jbfavre.org> <4D340504.7020203@jbfavre.org> <4D344AF4.80301@jbfavre.org> <4D3AB003.3040603@jbfavre.org> <20110127202755.GA4194@dumpdata.com> <4D41E7EE.4060502@jbfavre.org> <4D42E520.9020107@jbfavre.org> <1296560086.13091.131.camel@zakaz.uk.xensource.com> <4D47F9CF.2040107@jbfavre.org> <1296566401.13091.171.camel@zakaz.uk.xensource.com> <4D4814CE.5050303@jbfavre.org> <1296569931.13091.194.camel@zakaz.uk.xensource.com> <4D48234F.2020907@jbfavre.org> Reply-To: xen-devel@lists.xensource.com Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <4D48234F.2020907@jbfavre.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Hello, I think I'll have to read another time "man modprobe" :) See bellow, I've good news Le 01/02/2011 16:14, Jean Baptiste Favre a =C3=A9crit : > Le 01/02/2011 15:18, Ian Campbell a =C3=A9crit : > > On Tue, 2011-02-01 at 14:12 +0000, Jean Baptiste Favre wrote: > > > >> Le 01/02/2011 14:20, Ian Campbell a =C3=A9crit : > > > >>>>> If you restrict dom0 to >256MB but <512MB (using dom0_mem=3D on > >>> hypervisor > >>>>> command line) does the NIC work correctly in non-passedthrough fo= rm? > >>>> My Xen hypervisor commandline is as follow: > >>>> placeholder dom0_mem=3D256M dom0_max_vcpus=3D1 dom0_vcpus_pin logl= vl=3Dall > >>>> guest_loglvl=3Dall com1=3D115200,8n1 console=3Dcom1 > >>>> > >>>> Everything works great without passthrough, but my dom0 is 64bits > > which > >>>> may explain that (I do have this strange behaviour only with 32bit= s > >>>> kernels). > >>> > >>> I don't suppose you could try installing a 32 bit OS in dom0? (e.g. > > as a > >>> skanky hack you might fit something into your existing swap > >>> partition ;-)) > >> I could try, but that would means breaking my test platform. Let's > say I > >> prefer complete other tests before :) > > > > Sure, no problem. > > > >>>> I did not tried changing dom0_mem param. > >>>> > >>>>> Similarly does the kernel running native with mem=3D cause the > failure? > >>>> Not sure I understand what you mean here. > >>> > >>> I meant to run the system as a native (non-Xen) system and use the > >>> kernels mem=3D command line parameter to restrict the system to the > >>> problematic amounts of memory. Really just trying to verify if > > something > >>> is up specifically due to Xen or not. Probably needs a 32 bit > >>> user/kernel to be a useful experiment. > >> Ah ok. But then, running 32bits kernel with 64 bits OS will work ? (= I > >> didn't know 64bits kernel worked with 32bits OS before Konrad told m= e) > > > > 32 user on 64 kernel works, 64 user on 32 kernel does not so this wil= l > > tie in with the 32 bit test above too. > > > >>> What do "ifconfig" or "ethtool -S" show for the device after some > >>> failures. Do any of the numbers increment inline with the dropped > >>> frames? > >>> > >>> Perhaps interestingly 85 bytes, plus the ICMP, IP and Ethernet head= er > >>> sizes is something around 128 bytes (depending on options etc). > This is > >>> pure numerology but I notice that sky2 has a copybreak parameter > >>> ("Receive copy threshold") which defaults to 128. I think it would = be > >>> worth trying 64 and 256. > >> That's exactly the problem I faced with 256mb memory for my domU. > >> Outgoing packets reached my gateway (tcpdump saw them on it, as well= as > >> replies) but incoming packets greater than 128bits were blocked > >> somewhere between Xen hypervisor and domU user space (where I was > >> listening traffic with tcpdump). > >> > >> I didn't notice the copybreak from sky2. Where can I change it ? It > >> could be exactly what we're looking for from the beginning, couldn't > > it ? > >> How can I change it ? > > > > Assuming the driver is modular: > > "modprobe sky2 copybreak=3D" > > > > Depending on your distro there will be somewhere in /etc you can add > > this. e.g. on Debian you can create a file in /etc/modprobe.d/ > > containing "option sky copybreak=3D" other distros > > use /etc/modprobe.conf etc. > OK I see but it doesn't seems to have any effect. > I tried "option sky copybreak=3D0" to get all packet copied with no cha= nge. > > But I have to say that I'm a bit confused: as I run a PV domU, kernel > and initrd are provided by dom0. > So basically, I had no module related binaries installed. After > installation, I tried to remove module and reload it with different > configuration without changes. > Is there any way to provide this sort of option in kernel commandline s= o > that it 'll be taken into account even in initrd ? > > >>> Are you able to see any traffic with tcpdump, either in guest and/o= r > >>> from another host (may require switch configuration to allow it to = see > >>> the traffic). e.g. do you see the ICMP Echo Request but not the Ech= o > >>> Response or do you see nothing at all etc. > >> tcpdump tests have been done from my gateway and from domU. > >> On the GW: can see all incoming packets (whatever could be the > size) and > >> send all replies. > >> On the DomU: can see all outgoing packets but only incoming ones > shorter > >> than 128bytes (global size, means "ping -s85" and less) > > > > OK, so it is the sky2 receive path which is at fault, that's very use= ful > > info. It corresponds with the copybreak theory too. OK, just found it: after domU boot: - log in - ping -s 86 10.0.0.1 (fails) - rmmod sky2 - modprobe sky2 copybreak=3D0 (no packet copied) - ping -s 86 10.0.0.1 (works) So it's clearly related to that option. Now the question: what am I supposed to do ? It seems that adding this options in /etc/modprobe.d/sky2.conf does not work, neither using /etc/modules Regards, JB