From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
Jeremy Fitzhardinge <jeremy@goop.org>,
Jeremy Fitzhardinge <Jeremy.Fitzhardinge@citrix.com>
Subject: Re: [PATCH 09/22] xen: Find an unbound irq number in reverse order (high to low).
Date: Wed, 6 Oct 2010 18:59:56 -0400 [thread overview]
Message-ID: <20101006225956.GB20211@dumpdata.com> (raw)
In-Reply-To: <alpine.DEB.2.00.1010051617160.2440@kaball-desktop>
>
> Unfortunately this is the wrong way to fix the issue: Xen has a range of
> allowed pirq for each domain and we don't know exactly what is the
> maximum pirq (see my patch "xen: get the maximum number of pirqs from
> xen" [1]).
> Considering that we might use the irq number returned by
> find_unbound_irq through xen_allocate_pirq as pirq number in some cases,
Ah, but we wouldn't! We would end up only using the 'find_unbound_irq' for
event channels. For IRQs that are for physical devices (either being
real devices passed in or QEMU PCI devices) we end up requesting an IRQ that
matches whatever the device has defined in dev->irq (or whatever the
vectors values for MSI/MSI-X devices that is provided) via the Xen PCI frontend
driver (in case of QEMU whatever its emulation provides).
> starting from the highest value could be unsafe.
> In practice it should be impossible to see this issue because it can
> only happen if the irq returned by xen_allocate_pirq is higher than the
> max pirq in xen. However AFAIK when we call xen_allocate_pirq with the
> intention of using the return value as pirq we always fall in the if
> (identity_mapped_irq(gsi) || !xen_initial_domain()) that avoid calling
> find_unbound_irq.
Right, and we end up using an the pirq/gsi number at that point. This
patch would not touch that logic.
> In any case I still think that it is a good idea to try to avoid the problem.
>
> In order to solve this issue you can:
>
> - add a static offset to get_nr_hw_irqs in case of pv domains;
>
> - use the patch "xen: get the maximum number of pirqs from
> xen" [1] to get the highest possible pirq number from xen so that we can be
> sure we are using an allowed value;
>
> - if the xen interface allows it, disjoin the concept of pirq from the
> concept of linux irq; then you can let xen choose any pirq number it
> wants (see my patch "xen: support pirq != irq" [2] and how you can exploit
> it in the patch "xen: implement xen_hvm_register_pirq" [3]).
> Of course this doesn't help if it must be linux the one that chooses the
> pirq number.
All of those deal with PCI devices. This patch is intended for the bug
where we want to deal with event channels and we don't want them to
usurp the irq_desc that are above NR_IRQs and where we get a PCI device for
which its IRQ falls within what info_for_irq(irq) would return: IRQT_VIRQ or
IRQT_IPI or IRQT_EVTCHN.
In other words, avoid colliding with event channels b/c those are allocated
earlier than the PCI devices.
FYI, the reason this is not triggered under HVM is that it has an IOAPIC.
But under PV no such beast. Here is an output from /proc/interrupts:
A) HVM guest (2.6.36-rc6 w/ PCI passthrough):
CPU0 CPU1
0: 191 0 IO-APIC-edge timer
1: 4 3 IO-APIC-edge i8042
4: 46 25 IO-APIC-edge serial
8: 0 1 IO-APIC-edge rtc0
9: 0 0 IO-APIC-fasteoi acpi
12: 54 45 IO-APIC-edge i8042
14: 0 0 IO-APIC-edge ata_piix
15: 20 17 IO-APIC-edge ata_piix
16: 10670 0 xen-percpu-virq timer0
17: 0 9252 xen-percpu-virq timer1
18: 138 0 xen-dyn-event xenbus
19: 0 0 IO-APIC-fasteoi uhci_hcd:usb4
21: 0 0 IO-APIC-fasteoi ehci_hcd:usb1
40: 0 0 IO-APIC-fasteoi uhci_hcd:usb2
45: 0 0 IO-APIC-fasteoi uhci_hcd:usb3
64: 30 31 PCI-MSI-edge eth1
..snip..
B) PV guest (2.6.36-rc6 w/ PCI passthrough)
CPU0 CPU1 CPU2 CPU3
0: 1760 0 0 0 xen-percpu-virq timer0
1: 2 0 0 0 xen-percpu-ipi spinlock0
2: 2481 0 0 0 xen-percpu-ipi resched0
3: 23 0 0 0 xen-percpu-ipi callfunc0
4: 0 0 0 0 xen-percpu-virq debug0
5: 94 0 0 0 xen-percpu-ipi callfuncsingle0
6: 0 3474 0 0 xen-percpu-virq timer1
..snip..
C) PV guest (2.6.36-rc6 devel/xen-pcifront-0.8 with PCI devices.):
18: 0 0 0 0 xen-pirq-pcifront uhci_hcd:usb4
19: 0 0 0 0 xen-pirq-pcifront uhci_hcd:usb3
23: 0 0 0 0 xen-pirq-pcifront ehci_hcd:usb1, uhci_hcd:usb2
103: 38 0 0 0 xen-pirq-pcifront-msi eth1
276: 24 0 0 0 xen-dyn-event eth0
277: 188 0 0 0 xen-dyn-event hvc_console
278: 69 0 0 0 xen-dyn-event pcifront
279: 259 0 0 0 xen-dyn-event xenbus
280: 0 0 0 101 xen-percpu-ipi callfuncsingle3
... snip..
D) HVM 2.6.36-rc6 (devel/xen-pcifront-0.8)
0: 223 0 IO-APIC-edge timer
1: 3 4 IO-APIC-edge i8042
4: 44 53 IO-APIC-edge serial
8: 0 1 IO-APIC-edge rtc0
9: 0 0 IO-APIC-fasteoi acpi
12: 47 50 IO-APIC-edge i8042
14: 0 0 IO-APIC-edge ata_piix
15: 17 20 IO-APIC-edge ata_piix
19: 0 0 IO-APIC-fasteoi uhci_hcd:usb4
21: 0 0 IO-APIC-fasteoi ehci_hcd:usb1
40: 0 0 IO-APIC-fasteoi uhci_hcd:usb2
45: 0 0 IO-APIC-fasteoi uhci_hcd:usb3
64: 30 30 PCI-MSI-edge eth1
1149: 137 0 xen-dyn-event xenbus
1150: 0 8920 xen-percpu-virq timer1
1151: 11156 0 xen-percpu-virq timer0
..snip..
As you can see from B), the IRQs would be allocated from the start and inhibit
PCI devices from using them.
A) and D) show just the different off what this patch does.
>
> Feel free to pick up any of the patches I sent as part of the "PV on
> HVM: receive interrupts as xen events" series.
>
>
> References:
> [1] http://lkml.org/lkml/2010/8/30/177
> [2] http://lkml.org/lkml/2010/8/30/171
> [3] http://lkml.org/lkml/2010/8/30/172
next prev parent reply other threads:[~2010-10-06 23:00 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-10-04 18:13 [PATCH v7] Xen PCI + Xen PCI frontend driver Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 01/22] xen: Don't disable the I/O space Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 02/22] xen: define BIOVEC_PHYS_MERGEABLE() Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 03/22] xen: implement pirq type event channels Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 04/22] x86/io_apic: add get_nr_irqs_gsi() Konrad Rzeszutek Wilk
2010-10-05 9:13 ` Thomas Gleixner
2010-10-04 18:13 ` [PATCH 05/22] xen: identity map gsi->irqs Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 07/22] xen: set pirq name to something useful Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 08/22] xen: statically initialize cpu_evtchn_mask_p Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 09/22] xen: Find an unbound irq number in reverse order (high to low) Konrad Rzeszutek Wilk
2010-10-05 16:46 ` Stefano Stabellini
2010-10-06 22:59 ` Konrad Rzeszutek Wilk [this message]
2010-10-07 10:38 ` Stefano Stabellini
2010-10-07 14:05 ` [Xen-devel] " Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 10/22] xen: Provide a variant of xen_poll_irq with timeout Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 12/22] x86/PCI: Clean up pci_cache_line_size Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 13/22] x86/PCI: make sure _PAGE_IOMAP it set on pci mappings Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 14/22] x86/PCI: Export pci_walk_bus function Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 15/22] x86: Copy-n-paste arch_teardown_msi_irqs from msi.c to io_apic.c Konrad Rzeszutek Wilk
2010-10-05 9:07 ` Thomas Gleixner
2010-10-07 1:09 ` Konrad Rzeszutek Wilk
2010-10-07 2:57 ` Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 16/22] x86: Introduce x86_msi_ops Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 17/22] xen/x86/PCI: Add support for the Xen PCI subsystem Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 21/22] xen/pci: Request ACS when Xen-SWIOTLB is activated Konrad Rzeszutek Wilk
2010-10-04 18:13 ` [PATCH 22/22] MAINTAINERS: Add myself for Xen PCI and Xen SWIOTLB maintainer Konrad Rzeszutek Wilk
2010-10-05 7:47 ` [Xen-devel] [PATCH v7] Xen PCI + Xen PCI frontend driver Jan Beulich
2010-10-06 22:50 ` Konrad Rzeszutek Wilk
2010-10-07 19:44 ` Konrad Rzeszutek Wilk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101006225956.GB20211@dumpdata.com \
--to=konrad.wilk@oracle.com \
--cc=Jeremy.Fitzhardinge@citrix.com \
--cc=jeremy@goop.org \
--cc=linux-kernel@vger.kernel.org \
--cc=stefano.stabellini@eu.citrix.com \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox