From mboxrd@z Thu Jan 1 00:00:00 1970 From: Manish Jaggi Subject: Re: Xen/arm: Virtual ITS command queue handling Date: Thu, 21 May 2015 05:37:51 -0700 Message-ID: <555DD19F.3060600@caviumnetworks.com> References: <1431442942.8263.175.camel@citrix.com> <555239ED.2090400@citrix.com> <1431523416.8263.273.camel@citrix.com> <55535F30.7040903@citrix.com> <1431687548.8943.74.camel@citrix.com> <5555E46F.4010209@citrix.com> <1431694697.8943.119.camel@citrix.com> <5555F383.7030909@citrix.com> <1432037642.12989.106.camel@citrix.com> <555B3A5C.4090502@citrix.com> <1432042604.12989.124.camel@citrix.com> <1432043681.12989.135.camel@citrix.com> <1432045085.12989.146.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1432045085.12989.146.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell , Vijay Kilari Cc: Stefano Stabellini , Prasun Kapoor , manish.jaggi@caviumnetworks.com, Julien Grall , "xen-devel@lists.xen.org" , Julien Grall , Stefano Stabellini List-Id: xen-devel@lists.xenproject.org On Tuesday 19 May 2015 07:18 AM, Ian Campbell wrote: > On Tue, 2015-05-19 at 19:34 +0530, Vijay Kilari wrote: >> On Tue, May 19, 2015 at 7:24 PM, Ian Campbell wrote: >>> On Tue, 2015-05-19 at 14:36 +0100, Ian Campbell wrote: >>>> On Tue, 2015-05-19 at 14:27 +0100, Julien Grall wrote: >>>>> With the multiple vITS we would have to retrieve the number of vITS. >>>>> Maybe by extending the xen_arch_domainconfig? >>>> I'm sure we can find a way. >>>> >>>> The important question is whether we want to go for a N:N vits:pits >>>> mapping or 1:N. >>>> >>>> So far I think we are leaning (slightly?) towards the 1:N model, if we >>>> can come up with a satisfactory answer for what to do with global >>>> commands. >>> Actually, Julien just mentioned NUMA which I think is a strong argument >>> for the N:N model. >>> >>> We need to make a choice here one way or another, since it has knock on >>> effects on other parts, e.g the handling of SYNC and INVALL etc. >>> >>> Given that N:N seems likely to be simpler from the Xen side and in any >>> case doesn't preclude us moving to a 1:N model (or even a 2:N model etc) >>> in the future how about we start with that? >>> >>> If there is agreement in taking this direction then I will adjust the >>> relevant sections of the document to reflect this. >> Yes, this make Xen side simple. Most important point to discuss is >> >> 1) How Xen maps vITS to pITS. its0 -> vits0? > The choices are basically either Xen chooses and the tools get told (or > "Just Know" the result), or the tools choose and setup the mapping in > Xen via hypercalls. > This could be one possible flow: -1- xen code parses the pci node and creates a pci_hostbridge structure which stores the device_tree ptr. (using this pointer msi-parent (or respective its) can be retrieved) -2- dom0 invokes a hypercall to register pci_hostbridge (seg_no:cfg_addr) -3- Xen now knows that the device id (seg:bus:dev.fn) has which its. Using a helper function its node for a seg_no can be retrieved. -4- When a device is assigned to a domU, we introduce a new hypercall map_guest_bdf which would let xen know that for a guest how a virtual sbdf maps to a physical sdbf -5- domU is booted with a single virtual its node in device tree. Front end driver attaches this its as msi-parent -6- When domU accesses for ITS are trapped in Xen, using the helper function say get_phys_its_for_guest(guest_id, guest_sbdf, /*[out]*/its_ptr *its) its can be retrieved. AFAIK this is numa safe. >> 2) When PCI device is assigned to DomU, how does domU choose >> vITS to send commands. AFAIK, the BDF of assigned device >> is different from actual BDF in DomU. > AIUI this is described in the firmware tables. > > e.g. in DT via the msi-parent phandle on the PCI root complex or > individual device. > > Is there an assumption here that a single PCI root bridge is associated > with a single ITS block? Or can different devices on a PCI bus use > different ITS blocks? > > Ian. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel