From mboxrd@z Thu Jan 1 00:00:00 1970 From: Weidong Han Subject: Re: [PATCH] VT-d: improve RMRR validity checking Date: Wed, 10 Mar 2010 15:03:20 +0800 Message-ID: <4B974438.3050806@intel.com> References: <4B59098B.6000108@intel.com> <4B590FA4.4000008@jp.fujitsu.com> <4B59132B.40607@intel.com> <4B59188C.50901@jp.fujitsu.com> <4B59660F.4000909@intel.com> <7162ab21003091339i4adb8669safd5e074607386a2@mail.gmail.com> <4B9706B3.3050903@intel.com> <1268191101.3015.30.camel@2710p.home> <4B9711CF.7030908@intel.com> <1268192253.3015.37.camel@2710p.home> <4B971F57.9000903@intel.com> <1268196438.3015.56.camel@2710p.home> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1268196438.3015.56.camel@2710p.home> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Alex Williamson Cc: "xen-devel@lists.xensource.com" , Noboru Iwamatsu , "Cihula, Joseph" , "Kay, Allen M" , "linux@eikelenboom.it" , "keir.fraser@eu.citrix.com" List-Id: xen-devel@lists.xenproject.org Alex Williamson wrote: > On Wed, 2010-03-10 at 12:25 +0800, Weidong Han wrote: > >> Alex Williamson wrote: >> >>> On Wed, 2010-03-10 at 11:28 +0800, Weidong Han wrote: >>> >>> >>>> Alex Williamson wrote: >>>> >>>> >>>>> On Wed, 2010-03-10 at 10:40 +0800, Weidong Han wrote: >>>>> >>>>> >>>>> >>>>>> Alex Williamson wrote: >>>>>> >>>>>> >>>>>> >>>>>>> I have a system with what I consider to be a valid DRHD that's getting >>>>>>> tripped up on this patch. The problem is that the DRHD includes an >>>>>>> IOAPIC scope, where the IOAPIC is not materialized on the PCI bus. I >>>>>>> think Xen is being overzealous in it's validity checking and that this >>>>>>> is a valid configuration. What do others think? Are IOAPICs a >>>>>>> special case that we can allow to be non-existent on the PCI bus? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> Yes, IOAPIC can be not pci-discoverable. IOAPICs are only reported in >>>>>> the "Include_all" DRHD, and our patch won't check if the device is >>>>>> pci-discoverable or not for the "Include_all" DRHD. So I think the patch >>>>>> is no problem unless IOAPIC is not included in the "Include_all" DRHD. >>>>>> Can you post your boot logs? >>>>>> >>>>>> >>>>>> >>>>> Weidong, >>>>> >>>>> That's a very subtle restriction, and I'm not sure how it works in >>>>> practice. If I have a multi-IOH system, each with VT-d hardware, each >>>>> supporting interrupt remapping, each with one or more IOAPICs below >>>>> them, how can interrupt remapping work if we can only associate an >>>>> IOAPIC with the "include all" DRHD? I'm confused. Thanks, >>>>> >>>>> >>>>> >>>> Each IOH will have one "include all" DRHD which reports IOAPICs for each >>>> IOH. >>>> >>>> >>> Wouldn't that imply multiple PCI segments? The configuration I'm >>> looking at has multiple IOHs, all on the same PCI segment. By my >>> reading of the spec, we're only allowed to declare INCLUDE_PCI_ALL for >>> one DRHD within the segment. Am I incorrect? Thanks, >>> >>> >> Currently multiple PCI segments are not supported in Xen yet. So you >> encounter issue on multiple PCI segment system. We will support it after >> xen 4.0. >> > > > Which is exactly why we have multiple IOHs on the *same* PCI segment on > this system. How is it possible to support multiple IOHs, all on the > same PCI segment, each with VT-d hardware with interrupt remapping > support, each with one or more IOAPICs below them given the current > code? We cannot list the IOAPICs only under the INCLUDE_PCI_ALL DRHD > because that wouldn't provide the right information in the right place > for interrupt remapping on the other DRHDs. We cannot specify > INCLUDE_PCI_ALL on all of the DRHDs because the spec indicates we can > only have one INCLUDE_PCI_ALL DRHD per PCI segment (besides, we can't > have PCI sub-hierarchy scopes specified on INCLUDE_PCI_ALL DRHDs, which > means we'd have no way to associate PCI devices to a specific DRHD if > they all set this flag). > > I'm inclined to believe the hardware actually works correctly if we > associate an IOAPIC to a non-INCLUDE_PCI_ALL DRHD, but this validity > checking code prevents Xen from even trying to use it. > > Alex > This patch is no problem on our platform which has two IOHs, two IOAPICs. But there is only one DRHD, which is also INCLUDE_PCI_ALL DRHD. Can you post your Xen boot logs? Regards, Weidong