qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: David Woodhouse <dwmw2@infradead.org>
To: Joel Upham <jupham125@gmail.com>, qemu-devel@nongnu.org
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	Eduardo Habkost <eduardo@habkost.net>,
	 "Michael S. Tsirkin" <mst@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	"open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v1 01/23] pc/xen: Xen Q35 support: provide IRQ handling for PCI devices
Date: Tue, 29 Aug 2023 11:18:20 +0100	[thread overview]
Message-ID: <e31754530396548141e72e99d8d70470c2ea02df.camel@infradead.org> (raw)
In-Reply-To: <1c547c5581ce6192b70c68f39de108cdb2c73f7e.1687278381.git.jupham125@gmail.com>

[-- Attachment #1: Type: text/plain, Size: 3973 bytes --]

On Tue, 2023-06-20 at 13:24 -0400, Joel Upham wrote:
> The primary difference in PCI device IRQ management between Xen HVM and
> QEMU is that Xen PCI IRQs are "device-centric" while QEMU PCI IRQs are
> "chipset-centric". Namely, Xen uses PCI device BDF and INTx as coordinates
> to assert IRQ while QEMU finds out to which chipset PIRQ the IRQ is routed
> through the hierarchy of PCI buses and manages IRQ assertion on chipset
> side (as PIRQ inputs).
> 
> Two callback functions are used for this purpose: .map_irq and .set_irq
> (named after corresponding structure fields). Corresponding Xen-specific
> callback functions are piix3_set_irq() and pci_slot_get_pirq(). In Xen
> case these functions do not operate on pirq pin numbers. Instead, they use
> a specific value to pass BDF/INTx information between .map_irq and
> .set_irq -- PCI device devfn and INTx pin number are combined into
> pseudo-PIRQ in pci_slot_get_pirq, which piix3_set_irq later decodes back
> into devfn and INTx number for passing to *set_pci_intx_level() call.
> 
> For Xen on Q35 this scheme is still applicable, with the exception that
> function names are non-descriptive now and need to be renamed to show
> their common i440/Q35 nature. Proposed new names are:
> 
> xen_pci_slot_get_pirq --> xen_cmn_pci_slot_get_pirq
> xen_piix3_set_irq     --> xen_cmn_set_irq
> 
> Another IRQ-related difference between i440 and Q35 is the number of PIRQ
> inputs and PIRQ routers (PCI IRQ links in terms of ACPI) available. i440
> has 4 PCI interrupt links, while Q35 has 8 (PIRQA...PIRQH).
> Currently Xen have support for only 4 PCI links, so we describe only 4 of
> 8 PCI links in ACPI tables. Also, hvmloader disables PIRQ routing for
> PIRQE..PIRQH by writing 80h into corresponding PIRQ[n]_ROUT registers.
> 
> All this PCI interrupt routing stuff is largely an ancient legacy from PIC
> era. It's hardly worth to extend number of PCI links supported as we
> normally deal with APIC mode and/or MSI interrupts.
> 
> The only useful thing to do with PIRQE..PIRQH routing currently is to
> check if guest actually attempts to use it for some reason (despite ACPI
> PCI routing information provided). In this case, a warning is logged.
> 
> Things have changed a bit in modern Qemu, and more changes to the IRQ
> mapping had to be done inside the lpc_ich9 to write the irqs and setup
> the mappings.
> 
> Signed-off-by: Alexey Gerasimenko <x1917x@xxxxxxxxx>
> Signed-off-by: Joel Upham <jupham125@gmail.com>

Please ensure you test Linux guests with attached/emulated PCI devices
and (at least) the following command lines:

 • pci=nomsi
 • pci=nomsi noapic
 • xen_no_vector_callback pci=nomsi noapic
 • xen_no_vector_callback pci=nomsi

Obviously, with pci=nomsi you're actually *using* PCI INTx, and force
it not to use MSI.

With noapic you force it to use the i8259. The INTx should be routed to
a *different* interrupt number on the i8259 than the I/O APIC.
(I thought the IRQ should be asserted on both, but QEMU's
ich9_route_intx_pin_to_irq() seems to route to the I/O APIC *only* if
it's disabled in the i8259? Sadly, QEMU lacks a comment explaining if
that's a faithful emulation of the ICH9, or if it's just a workaround
for the fact that that function is only allowed one return value.)

This one has potential to be particularly entertaining because the
guest may route INTx IRQs to PIRQ event channels. And I think those are
supposed to use the I/O APIC pin numbering, *not* the PIC numbering. I
*think* you get away with it because of the workaround discussed above.

With xen_no_vector_callback you *also* use the emulated INTx of the Xen
platform PCI device, for event channel callbacks. We want to test that
via both the i8259 and the I/O APIC. And it also ensures that the INTx
of the real PCI devices is handled via the PIC or I/OAPIC instead of
letting the guest route them to PIRQs.





[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

  parent reply	other threads:[~2023-08-29 10:19 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-20 17:24 [PATCH v1 00/23] Q35 support for Xen Joel Upham
2023-06-20 17:24 ` [PATCH v1 01/23] pc/xen: Xen Q35 support: provide IRQ handling for PCI devices Joel Upham
2023-06-21  7:17   ` Daniel P. Berrangé
2023-06-21 16:49     ` Joel Upham
2023-08-29 10:18   ` David Woodhouse [this message]
2023-08-29 14:20   ` David Woodhouse
2023-06-20 17:24 ` [PATCH v1 02/23] pc/q35: Apply PCI bus BSEL property for Xen PCI device hotplug Joel Upham
2023-06-21 11:27   ` Igor Mammedov
2023-06-21 17:24     ` Joel Upham
2023-06-22  7:35       ` Igor Mammedov
2023-06-22 16:51         ` Julia Suvorova
2023-06-20 17:24 ` [PATCH v1 03/23] q35/acpi/xen: Provide ACPI PCI hotplug interface for Xen on Q35 Joel Upham
2023-06-21 11:28   ` Igor Mammedov
2023-06-21 17:27     ` Joel Upham
2023-06-20 17:24 ` [PATCH v1 04/23] q35/xen: Add Xen platform device support for Q35 Joel Upham
2023-06-20 17:24 ` [PATCH v1 05/23] q35: Fix incorrect values for PCIEXBAR masks Joel Upham
2023-06-20 17:24 ` [PATCH v1 06/23] xen/pt: XenHostPCIDevice: provide functions for PCI Capabilities and PCIe Extended Capabilities enumeration Joel Upham
2023-06-20 17:24 ` [PATCH v1 07/23] xen/pt: avoid reading PCIe device type and cap version multiple times Joel Upham
2023-06-20 17:24 ` [PATCH v1 08/23] xen/pt: determine the legacy/PCIe mode for a passed through device Joel Upham
2023-06-20 17:24 ` [PATCH v1 09/23] xen/pt: Xen PCIe passthrough support for Q35: bypass PCIe topology check Joel Upham
2023-06-20 17:24 ` [PATCH v1 10/23] xen/pt: add support for PCIe Extended Capabilities and larger config space Joel Upham
2023-06-20 17:24 ` [PATCH v1 11/23] xen/pt: handle PCIe Extended Capabilities Next register Joel Upham
2023-06-20 17:24 ` [PATCH v1 12/23] xen/pt: allow to hide PCIe Extended Capabilities Joel Upham
2023-06-20 17:24 ` [PATCH v1 13/23] xen/pt: add Vendor-specific PCIe Extended Capability descriptor and sizing Joel Upham
2023-06-20 17:24 ` [PATCH v1 14/23] xen/pt: add fixed-size PCIe Extended Capabilities descriptors Joel Upham
2023-06-20 17:24 ` [PATCH v1 15/23] xen/pt: add AER PCIe Extended Capability descriptor and sizing Joel Upham
2023-06-20 17:24 ` [PATCH v1 16/23] xen/pt: add descriptors and size calculation for RCLD/ACS/PMUX/DPA/MCAST/TPH/DPC PCIe Extended Capabilities Joel Upham
2023-06-20 17:24 ` [PATCH v1 17/23] xen/pt: add Resizable BAR PCIe Extended Capability descriptor and sizing Joel Upham
2023-06-20 17:24 ` [PATCH v1 18/23] xen/pt: add VC/VC9/MFVC PCIe Extended Capabilities descriptors " Joel Upham
2023-06-20 17:24 ` [PATCH v1 19/23] xen/pt: Fake capability id Joel Upham
2023-06-20 17:24 ` [PATCH v1 20/23] xen platform: unplug ahci object Joel Upham
2023-06-22  5:40   ` Bernhard Beschow
2023-10-19 12:37     ` David Woodhouse
2023-06-20 17:24 ` [PATCH v1 21/23] pc/q35: setup q35 for xen Joel Upham
2023-06-20 17:24 ` [PATCH v1 22/23] qdev-monitor/pt: bypass root device check Joel Upham
2023-06-20 17:24 ` [PATCH v1 23/23] s3 support: enabling s3 with q35 Joel Upham
2023-06-21 11:34   ` Igor Mammedov
2023-06-21 17:40     ` Joel Upham
2023-06-22 17:10 ` [PATCH v1 00/23] Q35 support for Xen Bernhard Beschow
2023-07-05 16:50   ` Joel Upham
2023-07-05 22:24     ` Bernhard Beschow
2023-08-22 14:18 ` Anthony PERARD
2023-08-22 17:15   ` Joel Upham

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e31754530396548141e72e99d8d70470c2ea02df.camel@infradead.org \
    --to=dwmw2@infradead.org \
    --cc=anthony.perard@citrix.com \
    --cc=eduardo@habkost.net \
    --cc=jupham125@gmail.com \
    --cc=marcel.apfelbaum@gmail.com \
    --cc=mst@redhat.com \
    --cc=paul@xen.org \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=richard.henderson@linaro.org \
    --cc=sstabellini@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).