From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Daney Subject: Re: PCI Pass-through in Xen ARM: Draft 4 Date: Tue, 15 Sep 2015 14:18:49 -0700 Message-ID: <55F88B39.1080206@caviumnetworks.com> References: <55CC668F.80104@caviumnetworks.com> <55F0D8FC.7050606@citrix.com> <55F8694B.6070904@cavium.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <55F8694B.6070904@cavium.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Jaggi, Manish" Cc: "Prasun.kapoor@cavium.com" , Ian Campbell , "Kumar, Vijaya" , =?ISO-2022-JP?B?GyRCIXobKEI=?= =?ISO-2022-JP?B?IFN0ZWZhbm8gU3RhYmVsbGluaQ==?= , "Daney, David" , Xen Devel , Julien Grall List-Id: xen-devel@lists.xenproject.org On 09/15/2015 11:58 AM, Jaggi, Manish wrote: > > > On Thursday 10 September 2015 06:42 AM, Julien Grall wrote: >> Hi Manish, > Hi Julien, sorry for late response.. >> >> On 13/08/2015 10:42, Manish Jaggi wrote: >>> 3.2. Mapping between streamID - deviceID - pci sbdf - requesterID >>> ----------------------------------------------------------------------------- >>> For a simpler case all should be equal to BDF. But there are some >>> devices >>> that use the wrong requester ID for DMA transactions. Linux kernel has >>> PCI >>> quirks for these. How the same be implemented in Xen or a diffrent >>> approach >>> has to be taken is TODO here. >>> >>> Till that time, for basic implementation it is assumed that all are >>> equal >>> to BDF. >> >> Back to this streamID = DeviceID = requestID = SBDF again... >> >> I've just found a patch for Linux send by one of your colleague about >> tweaking the requesterID for thunder-X board (See [1]). The most >> interesting bits are: >> >> static u32 pci_requester_id_ecam(struct pci_dev *dev) >> { >> return (((pci_domain_nr(dev->bus) >> 2) << 19) | >> ((pci_domain_nr(dev->bus) % 4) << 16) | >> (dev->bus->number << 8) | dev->devfn); >> } >> >> static u32 thunder_pci_requester_id(struct pci_dev *dev, u16 alias) >> { >> int ret; >> >> ret = thunder_pem_requester_id(dev); >> if (ret >= 0) >> return (u32)ret; >> >> return pci_requester_id_ecam(dev); >> } >> >> Which is then used to override the default function used by ITS to >> find the deviceID. >> >> AFAICT, this means that you can't safely assume that DeviceID == sBDF >> even for your platform. Is that right? > Yes. >> >> If so, I'm afraid that you have to handle DeviceID != sBDF (and so on) >> in your design doc. I.e how do you plan to get the base requester ID. >> > Right. But on our platform the requesterId is uniquely generated from > bdf. Adding David to confirm that. The requesterId is: Node << 19 + ECAM# << 16 + bdf For the Linux kernel, the Node/ECAM part is going to be read from OF device-tree or ACPI, not calculated with the above formula. >> I can see 2 different solutions: >> 1) Let DOM0 pass the first requester ID when registering the bus >> Pros: >> * Less per-platform code in Xen >> Cons: >> * Assume that the requester ID are contiguous. (Is it really a >> cons?) >> * Still require quirk for buggy device (i.e requester ID not >> correct) >> 2) Do it in Xen >> Pros: >> * We are not relying on DOM0 giving the requester ID >> => Not assuming contiguous requester ID >> Cons: >> * Per PCI bridge code to handle the mapping >> > We can have (3) that when PHYSDEVOP_pci_add_device is called the sbdf > and requesterID both are passed in hypercall. >> Regards, >> >> [1] https://lkml.org/lkml/2015/7/15/703 >> >> >> >