Re: [Xen-devel] [v1 1/2] xen/p2m: Create identity mappings for PFNs beyound E820 and PCI BARs

Gordan Bobic
2013-10-29T16:09:01Z

On Tue, 29 Oct 2013 11:11:29 -0400, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Tue, Oct 29, 2013 at 02:55:13PM +0000, Jan Beulich wrote:
>> >>> On 29.10.13 at 15:45, Konrad Rzeszutek Wilk
>> <konrad.wilk@oracle.com> wrote:
>> > On Tue, Oct 29, 2013 at 08:23:30AM +0000, Jan Beulich wrote:
>> >> >>> On 28.10.13 at 17:58, Konrad Rzeszutek Wilk
>> <konrad.wilk@oracle.com> wrote:
>> >> > On Fri, Oct 25, 2013 at 04:08:19PM -0600, Bjorn Helgaas wrote:
>> >> >> If you can look at PCI host bridge apertures instead of BARs,
>> that
>> >> >> would solve both problems. Reassigning those apertures is
>> >> >> theoretically possible but is not even a gleam in our eyes
>> yet.
>> >> >
>> >> > <nods> I think I have to have both (BARs and host bridge
>> apertures) as when
>> >> > we do PCI passthrough to a guest - we might do it without a
>> bridge.
>> >>
>> >> Why? Aren't the host bridge ranges necessarily a superset of the
>> >> individual devices' BARs?
>> >
>> > Yes. But when you pass in a PCI device to a PV guest you don't
>> pass in the
>> > bridge. Just the PCI device itself.
>>
>> Right you are. Which means that basing the whole logic on the
>> PCI device BARs is likely wrong anyway, not just because it
>> doesn't account for other MMIO ranges.
>
> Right, but that is OK. When you pass in a PCI device to a PV guest
> you
> only care about that specific device driver being able to access its
> BARs.

Maybe I'm reading this wrong, but my understanding is that you cannot
pass PCI bridges to domU anyway (in HVM - I tried). Is that not the
case?
I'm particularly interested in this for two reasons:

1) Some GPUs (Nvidia?) use bus resets to reset the GPU

2) Multi-GPU cards (e.g. GTX690/Grid K2) come with a PCIe bridge
of their own. I am successfully passing a modified GTX680 (as either
a Grid K2 or Quadro K5000) to a domU, but have completely failed to
get a modified GTX690 (Grid K2, exact same GPU as the GTX680) to work
with passthru. The only theory I have is that the extra PCIe bridge
is the problem (possibly a compound problem that only manigests when
there is a PLX (as per GTX690) PCIe bridge behind a NF200 PCIe bridge,
behind an Intel PCIe bridge.

Gordan