From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: [PATCH] VT-d: improve RMRR validity checking Date: Tue, 09 Mar 2010 21:47:18 -0700 Message-ID: <1268196438.3015.56.camel@2710p.home> References: <4B59098B.6000108@intel.com> <4B590FA4.4000008@jp.fujitsu.com> <4B59132B.40607@intel.com> <4B59188C.50901@jp.fujitsu.com> <4B59660F.4000909@intel.com> <7162ab21003091339i4adb8669safd5e074607386a2@mail.gmail.com> <4B9706B3.3050903@intel.com> <1268191101.3015.30.camel@2710p.home> <4B9711CF.7030908@intel.com> <1268192253.3015.37.camel@2710p.home> <4B971F57.9000903@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4B971F57.9000903@intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Weidong Han Cc: "xen-devel@lists.xensource.com" , Noboru Iwamatsu , "Cihula, Joseph" , "Kay, Allen M" , "linux@eikelenboom.it" , "keir.fraser@eu.citrix.com" List-Id: xen-devel@lists.xenproject.org On Wed, 2010-03-10 at 12:25 +0800, Weidong Han wrote: > Alex Williamson wrote: > > On Wed, 2010-03-10 at 11:28 +0800, Weidong Han wrote: > > > >> Alex Williamson wrote: > >> > >>> On Wed, 2010-03-10 at 10:40 +0800, Weidong Han wrote: > >>> > >>> > >>>> Alex Williamson wrote: > >>>> > >>>> > >>>>> I have a system with what I consider to be a valid DRHD that's getting > >>>>> tripped up on this patch. The problem is that the DRHD includes an > >>>>> IOAPIC scope, where the IOAPIC is not materialized on the PCI bus. I > >>>>> think Xen is being overzealous in it's validity checking and that this > >>>>> is a valid configuration. What do others think? Are IOAPICs a > >>>>> special case that we can allow to be non-existent on the PCI bus? > >>>>> > >>>>> > >>>>> > >>>> Yes, IOAPIC can be not pci-discoverable. IOAPICs are only reported in > >>>> the "Include_all" DRHD, and our patch won't check if the device is > >>>> pci-discoverable or not for the "Include_all" DRHD. So I think the patch > >>>> is no problem unless IOAPIC is not included in the "Include_all" DRHD. > >>>> Can you post your boot logs? > >>>> > >>>> > >>> Weidong, > >>> > >>> That's a very subtle restriction, and I'm not sure how it works in > >>> practice. If I have a multi-IOH system, each with VT-d hardware, each > >>> supporting interrupt remapping, each with one or more IOAPICs below > >>> them, how can interrupt remapping work if we can only associate an > >>> IOAPIC with the "include all" DRHD? I'm confused. Thanks, > >>> > >>> > >> Each IOH will have one "include all" DRHD which reports IOAPICs for each > >> IOH. > >> > > > > Wouldn't that imply multiple PCI segments? The configuration I'm > > looking at has multiple IOHs, all on the same PCI segment. By my > > reading of the spec, we're only allowed to declare INCLUDE_PCI_ALL for > > one DRHD within the segment. Am I incorrect? Thanks, > > > > Currently multiple PCI segments are not supported in Xen yet. So you > encounter issue on multiple PCI segment system. We will support it after > xen 4.0. Which is exactly why we have multiple IOHs on the *same* PCI segment on this system. How is it possible to support multiple IOHs, all on the same PCI segment, each with VT-d hardware with interrupt remapping support, each with one or more IOAPICs below them given the current code? We cannot list the IOAPICs only under the INCLUDE_PCI_ALL DRHD because that wouldn't provide the right information in the right place for interrupt remapping on the other DRHDs. We cannot specify INCLUDE_PCI_ALL on all of the DRHDs because the spec indicates we can only have one INCLUDE_PCI_ALL DRHD per PCI segment (besides, we can't have PCI sub-hierarchy scopes specified on INCLUDE_PCI_ALL DRHDs, which means we'd have no way to associate PCI devices to a specific DRHD if they all set this flag). I'm inclined to believe the hardware actually works correctly if we associate an IOAPIC to a non-INCLUDE_PCI_ALL DRHD, but this validity checking code prevents Xen from even trying to use it. Alex