From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: KVM call minutes 2013-01-29 - Port I/O Date: Thu, 31 Jan 2013 23:11:15 +0200 Message-ID: <20130131211115.GA16941@redhat.com> References: <87vcae6ab6.fsf@codemonkey.ws> <1359579910.23274.31.camel@pasglop> <87a9rq5p0p.fsf@codemonkey.ws> <20130130222017.GE6544@redhat.com> <1359585125.23274.41.camel@pasglop> <20130130224919.GA8820@redhat.com> <1359586927.23274.62.camel@pasglop> <1359588510.11144.382.camel@bling.home> <20130131104921.GA520@redhat.com> <1359650043.2585.6.camel@ul30vt.home> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: KVM devel mailing list , Juan Quintela , Alexander Graf , qemu-devel , Alon Levy , Gerd Hoffmann , Anthony Liguori , qemu-ppc , David Gibson , Andreas =?iso-8859-1?Q?F=E4rber?= , =?iso-8859-1?Q?Herv=E9?= Poussineau To: Alex Williamson Return-path: Content-Disposition: inline In-Reply-To: <1359650043.2585.6.camel@ul30vt.home> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org List-Id: kvm.vger.kernel.org On Thu, Jan 31, 2013 at 09:34:03AM -0700, Alex Williamson wrote: > > On Thu, 2013-01-31 at 12:49 +0200, Michael S. Tsirkin wrote: > > On Wed, Jan 30, 2013 at 04:28:30PM -0700, Alex Williamson wrote: > > > On Thu, 2013-01-31 at 10:02 +1100, Benjamin Herrenschmidt wrote: > > > > On Thu, 2013-01-31 at 00:49 +0200, Michael S. Tsirkin wrote: > > > > > > In practice they do (VGA at least) > > > > > > > > > > > > >From a SW modelling standpoint, I don't think it's worth > > > > > differentiating > > > > > > PCI and PCIE. > > > > > > > > > > > > Cheers, > > > > > > Ben. > > > > > > > > > > Interesting. > > > > > Do you have such hardware? Could you please dump > > > > > the output of lspci -vv? > > > > > > > > Any ATI or nVidia card still supports hard decoding of VGA regions for > > > > the sake of legacy operating systems and BIOSes :-) I don't know about > > > > Intel but I suppose it's the same. > > > > > > For example: > > > > > > -[0000:00]-+-00.0 Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 p > > > +-04.0-[02]--+-00.0 Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350] > > > > > > 00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D) (prog-if 00 [Normal decode]) > > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- > > Latency: 0, Cache Line Size: 64 bytes > > > Bus: primary=00, secondary=02, subordinate=02, sec-latency=0 > > > I/O behind bridge: 0000c000-0000cfff > > > Memory behind bridge: fd100000-fd1fffff > > > Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff > > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- > > BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B- > > > ^^^^ > > > VGA+ (VGA Enable) indicates positive decode of 0x3b0 - 0x3bb, 0x3c0 - > > > 0x3df, and 0xa0000 - 0xbfff. Device 2:00.0 of course doesn't report > > > these "ISA" ranges as they're implicit in the VGA class code. > > > > OK but this appears behind a bridge. So the bridge configuration tells > > the root complex where to send accesses to the VGA. > > > > But qemu currently puts devices directly on root bus. > > > > And as far as I can tell when we present devices directly on bus 0, we > > pretend these are integrated in the root complex. The spec seems to > > say explicitly that root complex integrated devices should not use legacy > > addresses or support hotplug. So I would be surprised if such one > > appears in real world. > > > > Luckily guests do not seem to be worried as long as we use ACPI. > > Yes, in fact I just figured out last night that Windows is unhappy with > assigned PCI devices on bus 0 that claim to be an endpoint in their PCIe > capability rather than an integrated endpoint. We'll need to do extra > mangling of the PCIe capability to massage it into the guest visible > topology. For now, just put you device behind an express bridge. This breaks acpi hotplug for now, but I'm looking into hotplug with bridges anyway. If you really need it I can give you a hack for hotplug too. Of course express does not allow hotplug of root complex parts but happens to work because we use ACPI. > Section 1.3.2.3 of the 3.0 spec says integrated endpoints must not > require I/O resources claimed through BAR(s). VGA skirts around this by > not having the legacy resources claimed by BARs, but instead being > implicit. Aha. I missed this point. > Are there other sections restricting legacy I/O? One other interesting things is that VGA enable bit (for bridge control register) does not appear in express spec at all. > It's common that a plugin VGA card sits behind a root port where the > bridge registers tell us about VGA routing, > but integrated VGA devices > are often on bus 0 though, here's an example: > > -[0000:00]-+-00.0 Intel Corporation 2nd Generation Core Processor Family DRAM Controller > +-02.0 Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller > > Often these systems will disable the integrated graphics when a plugin > graphics is installed below a root port. I'm not sure how the system > knows to route VGA to the integrated device vs the root port otherwise. I am guessing it disables the integrated graphics? > Here's a more interesting example: > > -+-[0000:01]-+-00.0 NVIDIA Corporation GT218 [GeForce G210M] > | \-00.1 NVIDIA Corporation High Definition Audio Controller > \-[0000:00]-+-00.0 Intel Corporation Mobile 4 Series Chipset Memory Controller Hub > +-01.0 Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port > > This system seems to have two host bridges with VGA behind each of them. > There's no bridge to control VGA routing, so I don't know how the > selection is done. Is IO space disabled for the inactive card? Maybe that is how. > It's possible the g210m never sees legacy VGA > accesses in this mode. This bios has another mode which makes the g210m > the primary graphics and hides the integrated graphics, essentially the > same as I mention above with hiding integrated endpoint graphics when > plugin graphics are used. Thanks, > > Alex