From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42390) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XIFU0-0002zS-AV for qemu-devel@nongnu.org; Fri, 15 Aug 2014 07:15:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XIFTs-0002ox-NV for qemu-devel@nongnu.org; Fri, 15 Aug 2014 07:15:44 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:17344) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XIFTs-0002oY-HT for qemu-devel@nongnu.org; Fri, 15 Aug 2014 07:15:36 -0400 Message-ID: <1408101325.14053.41.camel@abi.no.oracle.com> From: Knut Omang Date: Fri, 15 Aug 2014 13:15:25 +0200 In-Reply-To: <1408077757.11008.89.camel@ori.omang.mine.nu> References: <1407740702-4086-1-git-send-email-tamlokveer@gmail.com> <20140814111511.GS31346@redhat.com> <53ECA71F.6070700@web.de> <1408077757.11008.89.camel@ori.omang.mine.nu> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v3 0/5] intel-iommu: introduce Intel IOMMU (VT-d) emulation to q35 chipset List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jan Kiszka Cc: "Michael S. Tsirkin" , Stefan Weil , Le Tan , qemu-devel@nongnu.org, Alex Williamson , Anthony Liguori , Paolo Bonzini On Fri, 2014-08-15 at 06:42 +0200, Knut Omang wrote: > On Thu, 2014-08-14 at 14:10 +0200, Jan Kiszka wrote: > > On 2014-08-14 13:15, Michael S. Tsirkin wrote: > > > On Mon, Aug 11, 2014 at 03:04:57PM +0800, Le Tan wrote: > > >> Hi, > > >> > > >> These patches are intended to introduce Intel IOMMU (VT-d) emulation to q35 > > >> chipset. The major job in these patches is to add support for emulating Intel > > >> IOMMU according to the VT-d specification, including basic responses to CSRs > > >> accesses, the logics of DMAR (DMA remapping) and DMA memory address > > >> translations. > > > > > > Thanks! > > > Looks very good overall, I noted some coding style issues - I didn't > > > bother reporting each issue in every place where it appears - reported > > > each issue once only, so please find and fix all instances of each > > > issue. > > > > BTW, because I was in urgent need for virtual test environment for > > Jailhouse, I hacked interrupt remapping on top of Le's patches: > > > > http://git.kiszka.org/?p=qemu.git;a=shortlog;h=refs/heads/queues/vtd-intremap > > > > The approach likely needs further discussions and refinements but it > > already allows me to work on top with our hypervisor, and also Linux. > > You can see from the last commit that Le's work made it pretty easy to > > build this on top. > > Le, > > I have tried Jan's branch with my device setup which consists of a > minimal q35 setup, an ioh3420 root port (specified as -device > ioh3420,slot=0 ) and a pcie device plugged into that root port, which > gives the following lscpi -t: > > -[0000:00]-+-00.0 > +-01.0 > +-02.0 > +-03.0-[01]----00.0 > +-04.0 > +-1f.0 > +-1f.2 > \-1f.3 > > All seems to work beautifully (I see the ISA bridge happily receive > translations) until the first DMA from my device model (at 1:00.0) > arrives, at which point I get: > > [ 1663.732413] dmar: DMAR:[DMA Write] Request device [00:03.0] fault addr fffa0000 > [ 1663.732413] DMAR:[fault reason 02] Present bit in context entry is clear > > I would have expected request device 01:00.0 for this. > It is not clear to me yet if this is a weakness of the implementation of > ioh3420 or the iommu. Just wanted to let you know right away in case you > can shed some light to it or it is an easy fix, > > The device uses pci_dma_rw with itself as device pointer. To verify my hypothesis: with this rude hack my device now works much better: @@ -774,6 +780,8 @@ static void iommu_translate(VTDAddressSpace *vtd_as, int bus_num, int devfn, is_fpd_set = ce.lo & VTD_CONTEXT_ENTRY_FPD; } else { ret_fr = dev_to_context_entry(s, bus_num, devfn, &ce); + if (ret_fr) + ret_fr = dev_to_context_entry(s, 1, 0, &ce); is_fpd_set = ce.lo & VTD_CONTEXT_ENTRY_FPD; if (ret_fr) { ret_fr = -ret_fr; Looking at how things look on hardware, multiple devices often receive overlapping DMA address ranges for different physical addresses. So if I understand the way this works, every requester ID would also need to have it's own unique VTDAddressSpace, as each pci device/function sees a unique DMA address space.. Knut