From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: pv-ops domU not working with MSI interrupts on Nehalem Date: Wed, 13 Oct 2010 17:46:13 -0400 Message-ID: <20101013214613.GA7153@dumpdata.com> References: <20101001211111.GA18244@dumpdata.com> <20101011214605.GA18201@dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Bruce Edge Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org On Wed, Oct 13, 2010 at 02:36:47PM -0700, Bruce Edge wrote: > On Mon, Oct 11, 2010 at 2:46 PM, Konrad Rzeszutek Wilk > wrote: > > On Mon, Oct 11, 2010 at 02:12:22PM -0700, Bruce Edge wrote: > >> On Fri, Oct 1, 2010 at 2:11 PM, Konrad Rzeszutek Wilk > >> wrote: > >> > On Mon, Sep 27, 2010 at 08:52:39AM -0700, Bruce Edge wrote: > >> >> One of our developers who is working on a tachyon driver is > >> >> complaining that the pvops domU kernel is not working for these M= SI > >> >> interrupts. > >> >> This is using the current head of xen/2.6.32.x on both a single > >> >> Nahelam 920 and a dual E5540. This behavior is consistent with Xe= n > >> >> 4.0.1, 4.0.2.rc1-pre and 4.1. > >> > > >> > > >> > I just checked on my SuperMicro X8DTN, this combination > >> > =A0- For Dom0, git commit fe999249 (2.6.32.18) > >> > =A0- For DomU, devel/xen-pcifront-0.6 or devel/xen-pcifront-0.7 > >> > =A0- For Hypervisor I used cs 21976, but found that the latest (22= 155) works too > >> > > >> > with which where I passed in PCI devices with legacy IRQ, MSI, and= MSI-X. I tried > >> > a combination of doing this with IOMMU (VT-d) and without - both c= ases these devices: > >> > > >> > 00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB= UHCI Controller #1 (rev 02) > >> > 00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB= UHCI Controller #2 (rev 02) > >> > 00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB= UHCI Controller #3 (rev 02) > >> > 00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB= 2 EHCI Controller #1 (rev 02) > >> > 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Eth= ernet Controller (Copper) (rev 06) > >> > 0a:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Net= work Connection (rev 02) > >> > > >> > worked just fine (either defining pci=3D["..."] or just using pci-= attach). > >> > > >> > But if I use the latest xen/next or xen/stable-2.6.32.x it does no= t look > >> > that happy :-( > >> > > >> > > >> > >> Konrad, > >> To try eliminate the remaining differences here, could you post your > >> dom0/domU config files? > > > > Sure. See attached >=20 > Konrad, > That made a big difference. Looks much better now. It's been kicked > over to several developers who have each got our tachyon driver > working a little bit better. Hmm, I didn't really try to do anything fancy with the configs. Any inklings of what config option might have caused all this headache? >=20 > Now the sticking point is an apparent limitation on the amount of > memory one can request using pci_map_single. It appears that we can > only ask for 256K or less. We need a 2MB DMA buffer. You can't use SG buffers? And chain them together and provide them to the device? A lot of other drivers do this ... > Is there some alternate mechanism for getting a larger physically > contiguous buffer under pvops? Why the contiguous requirement? >=20 > Thanks >=20 > -Bruce >=20 > >> I'd like to build the same kernels to get an apples->apples comparis= on. > >> > >> Also, could you include your grub info and domU cfg file? > >> > >> These may eliminate some of the remaining diffs in the configs and > >> show why your's works while mine does not. > >> > >> Thanks > >> > >> -Bruce > >