[PATCH v2] PCI: vmd: Disable bridge window for domain reset

[PATCH v2] PCI: vmd: Disable bridge window for domain reset

[Nirmal Patel]

During domain reset process we are accidentally enabling
the prefetchable memory by writing 0x0 to Prefetchable Memory
Base and Prefetchable Memory Limit registers. As a result certain
platforms failed to boot up.

When clearing Prefetchable Memory Base, Prefetchable Memory
Limit and Prefetchable Base Upper 32 bits, the prefetchable
memory range becomes 0x0-0x575000fffff. As a result the
prefetchable memory is enabled accidentally. There is a gap
between implementation and the PCI Express spec.

Here is the quote from section 7.5.1.3.9 of PCI Express Base 6.0 spec:

  The Prefetchable Memory Limit register must be programmed to a smaller
  value than the Prefetchable Memory Base register if there is no
  prefetchable memory on the secondary side of the bridge.

Write proper values to required Memory Base registers and follow
same the style as  pci_disable_bridge_window.

Signed-off-by: Nirmal Patel <nirmal.patel@linux.intel.com>
---
v1->v2: Follow same chain of operation as pci_disable_bridge_window
        and update commit log.
---
 drivers/pci/controller/vmd.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 769eedeb8802..ca9081be917d 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -526,8 +526,16 @@ static void vmd_domain_reset(struct vmd_dev *vmd)
 				     PCI_CLASS_BRIDGE_PCI))
 					continue;
 
-				memset_io(base + PCI_IO_BASE, 0,
-					  PCI_ROM_ADDRESS1 - PCI_IO_BASE);
+				writel(0, base + PCI_IO_BASE);
+				/* MMIO Base/Limit */
+				writel(0x0000fff0, base + PCI_MEMORY_BASE);
+
+				/* Prefetchable MMIO Base/Limit */
+				writel(0, base + PCI_PREF_LIMIT_UPPER32);
+				writel(0x0000fff0, base + PCI_PREF_MEMORY_BASE);
+				writel(0xffffffff, base + PCI_PREF_BASE_UPPER32);
+				writel(0, base + PCI_IO_BASE_UPPER16);
+				writeb(0, base + PCI_CAPABILITY_LIST);
 			}
 		}
 	}
-- 
2.31.1

Re: [PATCH v2] PCI: vmd: Disable bridge window for domain reset

[Lorenzo Pieralisi]

On Wed, Jul 19, 2023 at 04:56:10PM -0400, Nirmal Patel wrote:
> During domain reset process we are accidentally enabling
> the prefetchable memory by writing 0x0 to Prefetchable Memory
> Base and Prefetchable Memory Limit registers. As a result certain
> platforms failed to boot up.
> 
> When clearing Prefetchable Memory Base, Prefetchable Memory
> Limit and Prefetchable Base Upper 32 bits, the prefetchable
> memory range becomes 0x0-0x575000fffff.

I asked before, I am asking again. I assume you are describing
what happens while the code is executing memset_io() and the
Prefetchable Base Upper 32 bits have not been written yet.

That's a problem but most likely the issue is the end result,
with the prefetchable window set to 0x0-0x000fffff.

Please explain and reword the commit log accordingly.

> As a result the
> prefetchable memory is enabled accidentally. There is a gap
> between implementation and the PCI Express spec.

Define the gap and how it affects this commit or remove this sentence.

[...]

> Write proper values to required Memory Base registers and follow
> same the style as  pci_disable_bridge_window.

Replace this sentence with:

"Disable the bridge window by executing a sequence of operations
borrowed from pci_disable_bridge_window(), that comply with the
PCI specifications".

Thanks,
Lorenzo

> Signed-off-by: Nirmal Patel <nirmal.patel@linux.intel.com>
> ---
> v1->v2: Follow same chain of operation as pci_disable_bridge_window
>         and update commit log.
> ---
>  drivers/pci/controller/vmd.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index 769eedeb8802..ca9081be917d 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -526,8 +526,16 @@ static void vmd_domain_reset(struct vmd_dev *vmd)
>  				     PCI_CLASS_BRIDGE_PCI))
>  					continue;
>  
> -				memset_io(base + PCI_IO_BASE, 0,
> -					  PCI_ROM_ADDRESS1 - PCI_IO_BASE);
> +				writel(0, base + PCI_IO_BASE);
> +				/* MMIO Base/Limit */
> +				writel(0x0000fff0, base + PCI_MEMORY_BASE);
> +
> +				/* Prefetchable MMIO Base/Limit */
> +				writel(0, base + PCI_PREF_LIMIT_UPPER32);
> +				writel(0x0000fff0, base + PCI_PREF_MEMORY_BASE);
> +				writel(0xffffffff, base + PCI_PREF_BASE_UPPER32);
> +				writel(0, base + PCI_IO_BASE_UPPER16);
> +				writeb(0, base + PCI_CAPABILITY_LIST);
>  			}
>  		}
>  	}
> -- 
> 2.31.1
>