From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jean Baptiste Favre Subject: Re: PCI passthrough issue Date: Fri, 04 Feb 2011 09:43:01 +0100 Message-ID: <4D4BBC15.4080201@jbfavre.org> References: <4D47F9CF.2040107@jbfavre.org> <1296566401.13091.171.camel@zakaz.uk.xensource.com> <4D4814CE.5050303@jbfavre.org> <1296569931.13091.194.camel@zakaz.uk.xensource.com> <4D48234F.2020907@jbfavre.org> <4D4828D9.6090601@jbfavre.org> <1296577389.13091.288.camel@zakaz.uk.xensource.com> <4D488355.8010706@jbfavre.org> <1296638873.13091.315.camel@zakaz.uk.xensource.com> <4D4930F3.608@jbfavre.org> <20110202174250.GA8148@dumpdata.com> Reply-To: xen-devel@lists.xensource.com Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20110202174250.GA8148@dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Hello, Sorry for not answering earlier. I made a test with OpenWRT, applying patches bellow. DomU boots, can connect, but nothing changed. I still have incoming packets dropped without any console log. Looking deeper, I can see ARP request coming in on my GW, replies coming out. On the domU, arp command output is: # arp IP address HW type Flags HW address Mask Device 10.0.0.1 0x1 0x0 00:00:00:00:00:00 * e= th0 So, now I've an Layer 2 issue as well :( I'm still trying to make my Debian kernel working so that I can test on Debian too. Regards, JB >>From Konrad: diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..37c0631 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -2331,7 +2331,7 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, if (likely(skb)) { pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, length, PCI_DMA_FROMDEVICE); - skb_copy_from_linear_data(re->skb, skb->data, length); + skb_copy_from_linear_data(skb, re->skb->data, length); skb->ip_summed =3D re->skb->ip_summed; skb->csum =3D re->skb->csum; pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, And from Ian: diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..645d9e9 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -2339,6 +2339,7 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, re->skb->ip_summed =3D CHECKSUM_NONE; skb_put(skb, length); } + WARN(skb =3D=3D NULL, "sky2: receive_copy failed netdev_alloc_skb_ip_al= ign length %u\n", length); return skb; } @@ -2386,10 +2387,15 @@ static struct sk_buff *receive_new(struct sky2_port *sky2, nre.skb =3D sky2_rx_alloc(sky2); if (unlikely(!nre.skb)) + { + WARN(1, "sky2: receive_new failed sky2_rx_alloc\n"); goto nobuf; - + } if (sky2_rx_map_skb(sky2->hw->pdev, &nre, hdr_space)) + { + WARN(1, "sky2: receive_new failed sky2_rx_map_skb\n"); goto nomap; + } skb =3D re->skb; sky2_rx_unmap_skb(sky2->hw->pdev, re); diff --git a/net/core/dev.c b/net/core/dev.c index 54277df..9c99612 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1516,6 +1516,7 @@ int dev_forward_skb(struct net_device *dev, struct sk_buff *skb) if (unlikely(!(dev->flags & IFF_UP) || (skb->len > (dev->mtu + dev->hard_header_len + VLAN_HLEN)))) { + printk(KERN_CRIT "dev_forward_skb dropping skb\n"); atomic_long_inc(&dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -2700,6 +2701,7 @@ enqueue: local_irq_restore(flags); + printk(KERN_CRIT "enqueue_to_backlog dropping skb\n"); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -3125,6 +3127,7 @@ ncls: if (pt_prev) { ret =3D pt_prev->func(skb, skb->dev, pt_prev, orig_dev); } else { + printk(KERN_CRIT "__netif_receive_skb dropping skb proto %#x\n", type)= ; atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); /* Jamal, now you will not able to escape explaining Le 02/02/2011 18:42, Konrad Rzeszutek Wilk a =E9crit : > On Wed, Feb 02, 2011 at 11:24:51AM +0100, Jean Baptiste Favre wrote: >> Hello Ian, >> >> My domU config file: >> >> # cat /cluster/xen/xps-106.cfg >> kernel =3D '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' >> ramdisk =3D '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' >> builder =3D 'linux' >> memory =3D '398' >> vcpus =3D '1' >> cpus =3D '2' >> localtime =3D 0 >> serial =3D 'pty' >> boot =3D 'cdn' >> disk =3D [ 'drbd:xps-106,xvda,w' ] >> on_poweroff =3D 'destroy' >> on_reboot =3D 'restart' >> on_crash =3D 'restart' >> extra =3D "root=3D/dev/mapper/xps--106-root ro iommu=3Dsoft swiotlb=3D= force >> console=3Dhvc0 xencons=3Dtty" >> pci =3D [ '04:00.0' ] >> name =3D 'xps-106' >> hostname =3D 'xps-106.clichy.jbfavre.org' >> >> >> Le 02/02/2011 10:27, Ian Campbell a =E9crit : >>> On Tue, 2011-02-01 at 22:04 +0000, Jean Baptiste Favre wrote: >>>> Le 01/02/2011 17:23, Ian Campbell a =E9crit : >>> >>>>> I assume you are not seeing "rx mapping error" in your domU dmesg? = Did >>>>> you post a full guest console log at some point? Comparing the logs= for >>>>> the 256MB, 398MB and 512MB guest RAM case might be useful. >>>> No sure I've ever posted that logs. But I can redo my tests :) >>> >>> yes, please do that. >> Please find attached both console startup logs with 256M and 512M: >> 256M_domU_console_logs.txt >> 512M_domU_console_logs.txt >> >> For 512M, I saw some kernel CallTrace I can not explain. There are not >> present with 256M. >> >> For 398M memory, I can't even start domU : >> # xm create /cluster/xen/xps-106.cfg -c >> Using config file "/cluster/xen/xps-106.cfg". >> [215739.007871] pciback 0000:04:00.0: device has been assigned to >> another domain! Over-writting the ownership, but beware. >> Started domain xps-106 (id=3D23) >> (XEN) mm.c:798:d23 Non-privileged (23) attempt to map I/O space 000000= 00 >> (XEN) mm.c:4644:d23 ptwr_emulate: could not get_page_from_l1e() >> >> As I told you, I'm still using Debian 2.6.37 kernel because I've some >> problem to compile 2.6.32.27 from Jeremy's git repository. >> I hope I can get it compiled today so I'll be able to test with that >> kernel as well. >=20 > So I've tried this on my Abit IP-35 box which has a=20 >=20 > 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-= E Gigabit Ethernet Controller (rev 13) > Subsystem: ABIT Computer Corp. Device 1085 > Flags: bus master, fast devsel, latency 0, IRQ 29 > Memory at fdefc000 (64-bit, non-prefetchable) [size=3D16K] > I/O ports at be00 [size=3D256] > Expansion ROM at [disabled] > Capabilities: [48] Power Management version 3 > Capabilities: [50] Vital Product Data > Capabilities: [5c] MSI: Enable+ Count=3D1/1 Maskable- 64bit+ > Capabilities: [e0] Express Legacy Endpoint, MSI 00 > Capabilities: [100] Advanced Error Reporting > Kernel driver in use: sky2 > Kernel modules: sky2 >=20 > And when I launch this guest with a 32-bit DomU: > kernel=3D"/mnt/lab/latest/vmlinuz" > ramdisk=3D"/mnt/lab/latest/initramfs.cpio.gz" > extra=3D"console=3Dhvc0 debug iommu=3Dsoft" > memory=3D320 > vcpus=3D1 > cpu=3D'2' > on_crash=3D"preserve" > #vif =3D [ 'bridge=3Dswitch' ] > pci =3D ["04:00.0"] > vfb =3D [ 'vnc=3D1, vnclisten=3D0.0.0.0,vncunused=3D1'] >=20 > And played around with the 'extra' to add 'swiotlb=3Dforce'. >=20 > The moment I had 'swiotlb=3Dforce' I could not get any DHCP > address from the NIC. If I did not have 'swiotlb=3Dforce' it would > work just fine (can ping any size, etc, this is with 320MB) >=20 >=20 > For fun, I upped the memory (320->720) and kept 'swiotlb=3Dforce' in ef= fect. > Same effect: can't do DHCP. >=20 > I look to have a different issue than you, which is that whenever I use > swiotlb=3Dforce, things go haywire. >=20 > Fyi, this is what DomU tells me: >=20 > 12:22:41 # 9 :~/=20 >> dmesg |grep Memor > [ 0.000000] Memory: 145640k/335872k available (3731k kernel code, 18= 9784k reserved, 1565k data, 436k init, 0k highmem) >=20 > 12:22:56 # 10 :~/=20 >> uname -a > Linux (none) 2.6.38-rc2-00028-gf2a2d8b #2 SMP Wed Feb 2 12:10:25 EST 20= 11 i686 i686 i386 GNU/Linux > rnet driver. > [ 0.921406] udevd (1126): /proc/1126/oom_adj is deprecated, please u= se /proc/1126/oom_score_adj instead. > [ 0.984886] sky2: driver version 1.28 > [ 0.995595] sky2 0000:04:00.0: BAR 0: set to [mem 0xfdefc000-0xfdeff= fff 64bit] (PCI address [0xfdefc000-0xfdefffff]) > [ 0.995662] sky2 0000:04:00.0: BAR 2: set to [io 0xbe00-0xbeff] (PC= I address [0xbe00-0xbeff]) > [ 0.995697] sky2 0000:04:00.0: enabling device (0000 -> 0003) > [ 0.996440] sky2 0000:04:00.0: Xen PCI mapped GSI18 to IRQ27 >=20 > (This is the #master branch from git://git.kernel.org/pub/scm/linux/ker= nel/git/konrad/xen.git) >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >=20