RE: Re: free_irq_vector on ia64

[Alex Williamson <alex.williamson@hp.com>]

   Here's what I see on dom0 bootup if I revert xen-unstable.hg cset
12884 (plus a little debug code):

...
pciback 0000:01:02.1: seizing device  [port 1 of e1000]
pciback 0000:02:05.0: seizing device  [a tulip nic]
pciback 0000:16:05.0: seizing device  [another tulip nic]
...
(XEN) assign_irq_vector(-1)...
(XEN) assign_irq_vector(-1) = 59
GSI 76 (level, low) -> CPU 2 (0x0200) vector 59
ACPI: PCI Interrupt 0000:16:05.0[A] -> GSI 76 (level, low) -> IRQ 59
ACPI: PCI interrupt for device 0000:16:05.0 disabled
GSI 76 (level, low) -> CPU 2 (0x0200) vector 59 unregistered
(XEN) free_irq_vector(59)
...
(XEN) assign_irq_vector(-1)...
(XEN) assign_irq_vector(-1) = 59
GSI 28 (level, low) -> CPU 3 (0x0300) vector 59
ACPI: PCI Interrupt 0000:02:05.0[A] -> GSI 28 (level, low) -> IRQ 59
ACPI: PCI interrupt for device 0000:02:05.0 disabled
GSI 28 (level, low) -> CPU 3 (0x0300) vector 59 unregistered
(XEN) free_irq_vector(59)
...
(XEN) assign_irq_vector(-1)...
(XEN) assign_irq_vector(-1) = 59
GSI 32 (level, low) -> CPU 0 (0x0000) vector 59
ACPI: PCI Interrupt 0000:01:02.1[B] -> GSI 32 (level, low) -> IRQ 59
ACPI: PCI interrupt for device 0000:01:02.1 disabled
GSI 32 (level, low) -> CPU 0 (0x0000) vector 59 unregistered
(XEN) free_irq_vector(59)
...
(XEN) assign_irq_vector(-1)...
(XEN) assign_irq_vector(-1) = 59
GSI 19 (level, low) -> CPU 1 (0x0100) vector 59
ACPI: PCI Interrupt 0000:00:02.2[C] -> GSI 19 (level, low) -> IRQ 59
ehci_hcd 0000:00:02.2: EHCI Host Controller
ehci_hcd 0000:00:02.2: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:02.2: request interrupt 59 failed
ehci_hcd 0000:00:02.2: USB bus 1 deregistered
ACPI: PCI interrupt for device 0000:00:02.2 disabled
GSI 19 (level, low) -> CPU 1 (0x0100) vector 59 unregistered
(XEN) free_irq_vector(59)

   So, I think each of the assigns of the hidden devices is from the
pci_enable_device() call in pcistub_init_device().  This then
immediately calls pciback_reset_device() which frees the irq.  Note how
vector 59 gets tossed around the hidden devices, then ends up being
re-used for the USB device and it doesn't work (at least request_irq()
failed).  The order of device startup might have more to do with the
Oops on shutdown than anything else (maybe a bad error path in the usb
shutdown notifier chain).  There's a slightly scary comment in
pci_stub.c that we likely run afoul of if we reuse the interrupt vector:

    /* HACK: Force device (& ACPI) to determine what IRQ it's on - we
     * must do this here because pcibios_enable_device may specify
     * the pci device's true irq (and possibly its other resources)
     * if they differ from what's in the configuration space.
     * This makes the assumption that the device's resources won't
     * change after this point (otherwise this code may break!)
     */

When I run lspci on these devices, they all show up on IRQ 59.  So, not
calling free_irq_vector() is a bad hack, but it makes sure the interrupt
vector assigned to the hidden doesn't get recycled somewhere else.
Perhaps we need make sure pciback_reset_device() doesn't release the
vector, but it's not obvious how to do that.  Thanks,

	Alex

-- 
Alex Williamson                             HP Open Source & Linux Org.