Re: [v1 1/2] xen/p2m: Create identity mappings for PFNs beyound E820 and PCI BARs

[Gordan Bobic <gordan@bobich.net>]

 On Tue, 29 Oct 2013 11:11:29 -0400, Konrad Rzeszutek Wilk 
 <konrad.wilk@oracle.com> wrote:
> On Tue, Oct 29, 2013 at 02:55:13PM +0000, Jan Beulich wrote:
>> >>> On 29.10.13 at 15:45, Konrad Rzeszutek Wilk 
>> <konrad.wilk@oracle.com> wrote:
>> > On Tue, Oct 29, 2013 at 08:23:30AM +0000, Jan Beulich wrote:
>> >> >>> On 28.10.13 at 17:58, Konrad Rzeszutek Wilk 
>> <konrad.wilk@oracle.com> wrote:
>> >> > On Fri, Oct 25, 2013 at 04:08:19PM -0600, Bjorn Helgaas wrote:
>> >> >> If you can look at PCI host bridge apertures instead of BARs, 
>> that
>> >> >> would solve both problems.  Reassigning those apertures is
>> >> >> theoretically possible but is not even a gleam in our eyes 
>> yet.
>> >> >
>> >> > <nods> I think I have to have both (BARs and host bridge 
>> apertures) as when
>> >> > we do PCI passthrough to a guest - we might do it without a 
>> bridge.
>> >>
>> >> Why? Aren't the host bridge ranges necessarily a superset of the
>> >> individual devices' BARs?
>> >
>> > Yes. But when you pass in a PCI device to a PV guest you don't 
>> pass in the
>> > bridge. Just the PCI device itself.
>>
>> Right you are. Which means that basing the whole logic on the
>> PCI device BARs is likely wrong anyway, not just because it
>> doesn't account for other MMIO ranges.
>
> Right, but that is OK. When you pass in a PCI device to a PV guest 
> you
> only care about that specific device driver being able to access its 
> BARs.

 Maybe I'm reading this wrong, but my understanding is that you cannot
 pass PCI bridges to domU anyway (in HVM - I tried). Is that not the 
 case?
 I'm particularly interested in this for two reasons:

 1) Some GPUs (Nvidia?) use bus resets to reset the GPU

 2) Multi-GPU cards (e.g. GTX690/Grid K2) come with a PCIe bridge
 of their own. I am successfully passing a modified GTX680 (as either
 a Grid K2 or Quadro K5000) to a domU, but have completely failed to
 get a modified GTX690 (Grid K2, exact same GPU as the GTX680) to work
 with passthru. The only theory I have is that the extra PCIe bridge
 is the problem (possibly a compound problem that only manigests when
 there is a PLX (as per GTX690) PCIe bridge behind a NF200 PCIe bridge,
 behind an Intel PCIe bridge.

 Gordan