From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wei Wang2 Subject: Re: [PATCH] AMD IOMMU: Use global interrupt remapping table by default Date: Wed, 20 Jul 2011 14:34:57 +0200 Message-ID: <201107201434.58081.wei.wang2@amd.com> References: <200910281732.07420.wei.wang2@amd.com> <201107201152.41482.wei.wang2@amd.com> <1311159025.20648.185.camel@zakaz.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <1311159025.20648.185.camel@zakaz.uk.xensource.com> Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Ian Campbell Cc: George Dunlap , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Wednesday 20 July 2011 12:50:25 Ian Campbell wrote: > On Wed, 2011-07-20 at 10:52 +0100, Wei Wang2 wrote: > > On Tuesday 19 July 2011 16:14:31 George Dunlap wrote: > > > Wei, > > > > > > Can you be more specific about which BIOSes behave poorly with > > > per-device intremap tables, and why? > > > > We found that, in some case, SATA device uses different device ids for > > dma remapping and interrupt remapping. Some early BIOSes cannot handle > > this situation correctly, so if SATA uses device id for DMA to lookup > > device table entry for intremap table and if intremap table is per-devi= ce > > configured, SATA device won't get the right table. > > Was this issue present in production BIOSes or do you mean early as in > pre-production? IOW can we drop the support non-share remapping table > altogether or do we need to fix things in this mode to force the IDT to > be identical across CPUs (either by resharing the IDT in that case, ick, > or by enforcing that the contents are the same for devices with this > property)? > > OOI was the issue a confusion between the SATA PCI device and the legacy > PCI IDE facet of the same device? Yes, using shared intremap table is the workaround for this issue. Ideally,= =20 BIOS should create 2 IVRS entries for SATA devices in IDE combined mode, on= e=20 for DMA the other for interrupt. But this setup is not strict compatible wi= th=20 iommu specification. So recent BIOS should have IDE combined mode disabled = in=20 this case. So I believe that remove the global table is safe from now on. I= =20 could send patches. Thanks, Wei > > > The problem with a global intremap table is that, AFAICT, it's not > > > fundamentally compatible with per-cpu IDTs. With per-cpu IDTs, > > > different devices may end up with interrupts mapped to different cpus > > > but the same vector (i.e., device A mapped to cpu 9 vector 67, cpu B > > > mapped to cpu 12 vector 67). This is by design; the whole point of > > > the per-cpu IDTs is to avoid restricting the number of IRQs to the > > > number of vectors. But it seems that the intremap table only maps > > > vectors, not destination IDs; so in the example above, both devices' > > > interrupts would end up being remapped to the same place, causing one > > > device driver to get both sets of interrupts, and the other to get > > > none. > > > > Yes, obviously a problem...Using shared intremap table, devices uses the > > same vector and delivery mode will end up to the same remapping entry. = Is > > per-cpu IDTs enable by default in Xen? > > I didn't think it was even optional these days, but I didn't check. > > > > Do I understand correctly? If so, it seems like we should switch to > > > per-device intremap tables by default; and if we're using a global > > > intremap table, we need to somehow make sure that vectors are not > > > shared across cpus. > > > > I agree to use per-device table by default, since BIOS issue has been > > fixed and per-device table also has some security advantages. > > > > Thanks, > > Wei > > > > > -George > > > > > > On Wed, Oct 28, 2009 at 4:32 PM, Wei Wang2 wrote: > > > > Using a global interrupt remapping table shared by all devices has > > > > better compatibility with certain old BIOSes. Per-device interrupt > > > > remapping table can still be enabled by using a new parameter > > > > "amd-iommu-perdev-intremap". Thanks, > > > > Wei > > > > > > > > Signed-off-by: Wei Wang > > > > -- > > > > AMD GmbH, Germany > > > > Operating System Research Center > > > > > > > > Legal Information: > > > > Advanced Micro Devices GmbH > > > > Karl-Hammerschmidt-Str. 34 > > > > 85609 Dornach b. M=C3=BCnchen > > > > > > > > Gesch=C3=A4ftsf=C3=BChrer: Andrew Bowd, Thomas M. McCoy, Giuliano M= eroni > > > > Sitz: Dornach, Gemeinde Aschheim, Landkreis M=C3=BCnchen > > > > Registergericht M=C3=BCnchen, HRB Nr. 43632 > > > > > > > > _______________________________________________ > > > > Xen-devel mailing list > > > > Xen-devel@lists.xensource.com > > > > http://lists.xensource.com/xen-devel > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel