[patch 0/3] x86/PCI/MSI: Make sure that irq reservation mode works everywhere

[patch 0/3] x86/PCI/MSI: Make sure that irq reservation mode works everywhere

[Thomas Gleixner]

Dexuan reported that the recent rework of the vector allocation mode in x86
broke HyperV PCI passtrough because the rework missed to add the
MSI_FLAG_MUST_REACTIVATE flag to the HyperV/PCI interrupt domain info.

The simple solution would be to set the flag in the HyperV/PCI driver but
it's better to make this generic and let the PCI/MSI core code set the flag
when reservation mode is enabled. That ensures that future users of this
wont trip over the same problem.

Thanks,

	tglx

---
 arch/x86/Kconfig           |    2 +-
 arch/x86/kernel/apic/msi.c |    5 ++---
 drivers/pci/msi.c          |    2 ++
 kernel/irq/Kconfig         |    3 +++
 4 files changed, 8 insertions(+), 4 deletions(-)

[patch 1/3] genirq: Add config option for reservation mode

[Thomas Gleixner]

The interrupt reservation mode requires reactivation of PCI/MSI
interrupts. Create a config option, so the PCI code can set the
corresponding flag when required.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/irq/Kconfig |    3 +++
 1 file changed, 3 insertions(+)

--- a/kernel/irq/Kconfig
+++ b/kernel/irq/Kconfig
@@ -100,6 +100,9 @@ config IRQ_TIMINGS
 config GENERIC_IRQ_MATRIX_ALLOCATOR
 	bool
 
+config GENERIC_IRQ_RESERVATION_MODE
+	bool
+
 config IRQ_DOMAIN_DEBUG
 	bool "Expose hardware/virtual IRQ mapping via debugfs"
 	depends on IRQ_DOMAIN && DEBUG_FS

[patch 2/3] PCI/MSI: Set MSI_FLAG_MUST_REACTIVATE in core code

[Thomas Gleixner]

If interrupt reservation mode is enabled then the PCI/MSI interrupts must
be reactivated after early activation.

Make sure that all callers of pci_msi_create_irq_domain() have the
MSI_FLAG_MUST_REACTIVATE set when reservation mode is enabled.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 drivers/pci/msi.c |    2 ++
 1 file changed, 2 insertions(+)

--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -1441,6 +1441,8 @@ struct irq_domain *pci_msi_create_irq_do
 		pci_msi_domain_update_chip_ops(info);
 
 	info->flags |= MSI_FLAG_ACTIVATE_EARLY;
+	if (IS_ENABLED(CONFIG_GENERIC_IRQ_RESERVATION))
+		info->flags |= MSI_FLAG_MUST_REACTIVATE;
 
 	domain = msi_create_irq_domain(fwnode, info, parent);
 	if (!domain)

[patch 3/3] x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE

[Thomas Gleixner]

Select CONFIG_GENERIC_IRQ_RESERVATION_MODE so PCI/MSI domains get the
MSI_FLAG_MUST_REACTIVATE flag set in pci_msi_create_irq_domain().

Remove the explicit setters of this flag in the apic/msi code as they are
not longer required.

Fixes: 4900be83602b ("x86/vector/msi: Switch to global reservation mode")
Reported-by: Dexuan Cui <decui@microsoft.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/Kconfig           |    2 +-
 arch/x86/kernel/apic/msi.c |    5 ++---
 2 files changed, 3 insertions(+), 4 deletions(-)

--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -95,7 +95,7 @@ config X86
 	select GENERIC_IRQ_MATRIX_ALLOCATOR	if X86_LOCAL_APIC
 	select GENERIC_IRQ_MIGRATION		if SMP
 	select GENERIC_IRQ_PROBE
-	select GENERIC_IRQ_SHOW
+	select GENERIC_IRQ_RESERVATION_MODE
 	select GENERIC_PENDING_IRQ		if SMP
 	select GENERIC_SMP_IDLE_THREAD
 	select GENERIC_STRNCPY_FROM_USER
--- a/arch/x86/kernel/apic/msi.c
+++ b/arch/x86/kernel/apic/msi.c
@@ -129,7 +129,7 @@ static struct msi_domain_ops pci_msi_dom
 
 static struct msi_domain_info pci_msi_domain_info = {
 	.flags		= MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-			  MSI_FLAG_PCI_MSIX | MSI_FLAG_MUST_REACTIVATE,
+			  MSI_FLAG_PCI_MSIX,
 	.ops		= &pci_msi_domain_ops,
 	.chip		= &pci_msi_controller,
 	.handler	= handle_edge_irq,
@@ -167,8 +167,7 @@ static struct irq_chip pci_msi_ir_contro
 
 static struct msi_domain_info pci_msi_ir_domain_info = {
 	.flags		= MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-			  MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX |
-			  MSI_FLAG_MUST_REACTIVATE,
+			  MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX,
 	.ops		= &pci_msi_domain_ops,
 	.chip		= &pci_msi_ir_controller,
 	.handler	= handle_edge_irq,

Re: [patch 2/3] PCI/MSI: Set MSI_FLAG_MUST_REACTIVATE in core code

[Thomas Gleixner]

On Tue, 17 Oct 2017, Thomas Gleixner wrote:

> If interrupt reservation mode is enabled then the PCI/MSI interrupts must
> be reactivated after early activation.
> 
> Make sure that all callers of pci_msi_create_irq_domain() have the
> MSI_FLAG_MUST_REACTIVATE set when reservation mode is enabled.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  drivers/pci/msi.c |    2 ++
>  1 file changed, 2 insertions(+)
> 
> --- a/drivers/pci/msi.c
> +++ b/drivers/pci/msi.c
> @@ -1441,6 +1441,8 @@ struct irq_domain *pci_msi_create_irq_do
>  		pci_msi_domain_update_chip_ops(info);
>  
>  	info->flags |= MSI_FLAG_ACTIVATE_EARLY;
> +	if (IS_ENABLED(CONFIG_GENERIC_IRQ_RESERVATION))

Bah. quilt refresh seems overrated. Correct patch below.

Thanks,

	tglx

8<----------------

--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -1441,6 +1441,8 @@ struct irq_domain *pci_msi_create_irq_do
 		pci_msi_domain_update_chip_ops(info);
 
 	info->flags |= MSI_FLAG_ACTIVATE_EARLY;
+	if (IS_ENABLED(CONFIG_GENERIC_IRQ_RESERVATION_MODE))
+		info->flags |= MSI_FLAG_MUST_REACTIVATE;
 
 	domain = msi_create_irq_domain(fwnode, info, parent);
 	if (!domain)

RE: [patch 3/3] x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE

[Dexuan Cui]

> From: Thomas Gleixner [mailto:tglx@linutronix.de]
> Sent: Tuesday, October 17, 2017 12:55 AM
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -95,7 +95,7 @@ config X86
>  	select GENERIC_IRQ_MATRIX_ALLOCATOR	if X86_LOCAL_APIC
>  	select GENERIC_IRQ_MIGRATION		if SMP
>  	select GENERIC_IRQ_PROBE
> -	select GENERIC_IRQ_SHOW
> +	select GENERIC_IRQ_RESERVATION_MODE
>  	select GENERIC_PENDING_IRQ		if SMP
>  	select GENERIC_SMP_IDLE_THREAD
>  	select GENERIC_STRNCPY_FROM_USER

Hi Thomas,
It looks you removed GENERIC_IRQ_SHOW by accident?

I got this build error:
...
  LD      vmlinux.o
  MODPOST vmlinux.o
fs/proc/interrupts.o:(.rodata+0x118): undefined reference to `show_interrup=
ts'
Makefile:1000: recipe for target 'vmlinux' failed
make: *** [vmlinux] Error 1

Adding GENERIC_IRQ_SHOW back can fix the build failure, and with the
pachset Hyper-V PCI pass-through can work fine.

Thanks,
-- Dexuan

RE: [patch 3/3] x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE

[Thomas Gleixner]

On Tue, 17 Oct 2017, Dexuan Cui wrote:

> > From: Thomas Gleixner [mailto:tglx@linutronix.de]
> > Sent: Tuesday, October 17, 2017 12:55 AM
> > --- a/arch/x86/Kconfig
> > +++ b/arch/x86/Kconfig
> > @@ -95,7 +95,7 @@ config X86
> >  	select GENERIC_IRQ_MATRIX_ALLOCATOR	if X86_LOCAL_APIC
> >  	select GENERIC_IRQ_MIGRATION		if SMP
> >  	select GENERIC_IRQ_PROBE
> > -	select GENERIC_IRQ_SHOW
> > +	select GENERIC_IRQ_RESERVATION_MODE
> >  	select GENERIC_PENDING_IRQ		if SMP
> >  	select GENERIC_SMP_IDLE_THREAD
> >  	select GENERIC_STRNCPY_FROM_USER
> 
> Hi Thomas,
> It looks you removed GENERIC_IRQ_SHOW by accident?

Indeed. My patch foo today is more than bad....

> I got this build error:
> ...
>   LD      vmlinux.o
>   MODPOST vmlinux.o
> fs/proc/interrupts.o:(.rodata+0x118): undefined reference to `show_interrupts'
> Makefile:1000: recipe for target 'vmlinux' failed
> make: *** [vmlinux] Error 1
> 
> Adding GENERIC_IRQ_SHOW back can fix the build failure, and with the
> pachset Hyper-V PCI pass-through can work fine.

Thanks for testing.

       tglx

Re: [patch 0/3] x86/PCI/MSI: Make sure that irq reservation mode works everywhere

[Bjorn Helgaas]

On Tue, Oct 17, 2017 at 09:54:56AM +0200, Thomas Gleixner wrote:
> Dexuan reported that the recent rework of the vector allocation mode in x86
> broke HyperV PCI passtrough because the rework missed to add the
> MSI_FLAG_MUST_REACTIVATE flag to the HyperV/PCI interrupt domain info.
> 
> The simple solution would be to set the flag in the HyperV/PCI driver but
> it's better to make this generic and let the PCI/MSI core code set the flag
> when reservation mode is enabled. That ensures that future users of this
> wont trip over the same problem.
> 
> Thanks,
> 
> 	tglx
> 
> ---
>  arch/x86/Kconfig           |    2 +-
>  arch/x86/kernel/apic/msi.c |    5 ++---
>  drivers/pci/msi.c          |    2 ++
>  kernel/irq/Kconfig         |    3 +++
>  4 files changed, 8 insertions(+), 4 deletions(-)

This mentions 4900be83602b ("x86/vector/msi: Switch to global reservation
mode"), which I don't have, so I assume it's an x86 thing.  So I guess
you'll probably merge this via the same tree?

Here's my ack for the PCI part:

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

Re: [patch 0/3] x86/PCI/MSI: Make sure that irq reservation mode works everywhere

[Thomas Gleixner]

On Fri, 20 Oct 2017, Bjorn Helgaas wrote:
> On Tue, Oct 17, 2017 at 09:54:56AM +0200, Thomas Gleixner wrote:
> > Dexuan reported that the recent rework of the vector allocation mode in x86
> > broke HyperV PCI passtrough because the rework missed to add the
> > MSI_FLAG_MUST_REACTIVATE flag to the HyperV/PCI interrupt domain info.
> > 
> > The simple solution would be to set the flag in the HyperV/PCI driver but
> > it's better to make this generic and let the PCI/MSI core code set the flag
> > when reservation mode is enabled. That ensures that future users of this
> > wont trip over the same problem.
> > 
> > Thanks,
> > 
> > 	tglx
> > 
> > ---
> >  arch/x86/Kconfig           |    2 +-
> >  arch/x86/kernel/apic/msi.c |    5 ++---
> >  drivers/pci/msi.c          |    2 ++
> >  kernel/irq/Kconfig         |    3 +++
> >  4 files changed, 8 insertions(+), 4 deletions(-)
> 
> This mentions 4900be83602b ("x86/vector/msi: Switch to global reservation
> mode"), which I don't have, so I assume it's an x86 thing.  So I guess
> you'll probably merge this via the same tree?

Yes, it's in tip/x86/apic branch and the fixups are applied there now as well.

Thanks,

	tglx