From mboxrd@z Thu Jan  1 00:00:00 1970
From: Julien Grall <julien.grall@citrix.com>
Subject: Re: PCI Pass-through in Xen ARM: Draft 4
Date: Thu, 10 Sep 2015 02:12:28 +0100
Message-ID: <55F0D8FC.7050606@citrix.com>
References: <55CC668F.80104@caviumnetworks.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <55CC668F.80104@caviumnetworks.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Manish Jaggi <mjaggi@caviumnetworks.com>, Xen Devel <xen-devel@lists.xen.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, =?UTF-8?Q?=e2=98=85_Stefano_Stabellini?= <stefano.stabellini@eu.citrix.com>, =?UTF-8?Q?Manish_Jaggi=2c=e2=98=85_Kumar=2c_Vijaya?= <Vijaya.Kumar@caviumnetworks.com>, Julien Grall <julien.grall@linaro.org>, Ian Campbell <ian.campbell@citrix.com>
Cc: "Prasun.kapoor@cavium.com" <Prasun.kapoor@cavium.com>
List-Id: xen-devel@lists.xenproject.org

Hi Manish,

On 13/08/2015 10:42, Manish Jaggi wrote:
>   3.2.    Mapping between streamID - deviceID - pci sbdf - requesterID
>   -----------------------------------------------------------------------------
>   For a simpler case all should be equal to BDF. But there are some devices
>   that use the wrong requester ID for DMA transactions. Linux kernel has
> PCI
>   quirks for these. How the same be implemented in Xen or a diffrent
> approach
>   has to be taken is TODO here.
>
>   Till that time, for basic implementation it is assumed that all are equal
>   to BDF.

Back to this streamID = DeviceID = requestID = SBDF again...

I've just found a patch for Linux send by one of your colleague about 
tweaking the requesterID for thunder-X board (See [1]). The most 
interesting bits are:

static u32 pci_requester_id_ecam(struct pci_dev *dev)
{
	return (((pci_domain_nr(dev->bus) >> 2) << 19) |
		((pci_domain_nr(dev->bus) % 4) << 16) |
		(dev->bus->number << 8) | dev->devfn);
}

static u32 thunder_pci_requester_id(struct pci_dev *dev, u16 alias)
{
	int ret;

	ret = thunder_pem_requester_id(dev);
	if (ret >= 0)
		return (u32)ret;

	return pci_requester_id_ecam(dev);
}

Which is then used to override the default function used by ITS to find 
the deviceID.

AFAICT, this means that you can't safely assume that DeviceID == sBDF 
even for your platform. Is that right?

If so, I'm afraid that you have to handle DeviceID != sBDF (and so on) 
in your design doc. I.e how do you plan to get the base requester ID.

I can see 2 different solutions:
	1) Let DOM0 pass the first requester ID when registering the bus
	   Pros:
		* Less per-platform code in Xen
	   Cons:
		* Assume that the requester ID are contiguous. (Is it really a cons?)
		* Still require quirk for buggy device (i.e requester ID not correct)
	2) Do it in Xen
	   Pros:
		* We are not relying on DOM0 giving the requester ID
			=> Not assuming contiguous requester ID
  	   Cons:
		* Per PCI bridge code to handle the mapping

Regards,

[1] https://lkml.org/lkml/2015/7/15/703



-- 
Julien Grall