Re: [PATCH v4] PCI: endpoint: Add prefetch BAR support

[Niklas Cassel <cassel@kernel.org>]

On Wed, Feb 28, 2024 at 07:14:48PM +0530, Shradha Todi wrote:
> Reviewed-by: Shradha Todi <shradha.t@samsung.com>
> 
> This patch looks useful. Can we revisit this and get it merged?

Hello Shradha,

This patch is two years old, and no longer applies to pci-next.


However:
Usually, fixed hardware requirements are specified in
struct pci_epc_features (more specifically struct pci_epc_bar_desc).

A requested BAR configuration by an EPF is specified in struct epf_bar.


I don't think that Prefetch is a fixed hardware requirement,
so I do not think that we should put it in struct pci_epc_features.

It seems more like something that an endpoint function driver can
chose to request (or not to request), just like MEM_TYPE_64.

From the PCIe base spec:
"Generally only 64-bit BARs are good candidates, since only Legacy
Endpoints are permitted to set the Prefetchable bit in 32-bit BARs,
and most scalable platforms map all 32-bit Memory BARs into
non-prefetchable Memory Space regardless of the Prefetchable bit value."

"For a PCI Express Endpoint, 64-bit addressing must be supported for all BARs
that have the Prefetchable bit Set. 32-bit addressing is permitted for all BARs
that do not have the Prefetchable bit Set."

"Any device that has a range that behaves like normal memory should mark the
range as prefetchable. A linear frame buffer in a graphics device is an example
of a range that should be marked prefetchable."

We are not a legacy endpoint, so we should never set Prefetch for 32-bit BARs.
For 64-bit BARs, we should always set it, if the EPF-core allocated the memory
(regular memory) for that BAR.


Thus, I think the best solution is to do:

diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
index cd4ffb39dcdc..186c8cd87bb3 100644
--- a/drivers/pci/endpoint/functions/pci-epf-test.c
+++ b/drivers/pci/endpoint/functions/pci-epf-test.c
@@ -879,7 +879,8 @@ static void pci_epf_configure_bar(struct pci_epf *epf,
        for (i = 0; i < PCI_STD_NUM_BARS; i++) {
                epf_bar = &epf->bar[i];
                if (epc_features->bar[i].only_64bit)
-                       epf_bar->flags |= PCI_BASE_ADDRESS_MEM_TYPE_64;
+                       epf_bar->flags |= (PCI_BASE_ADDRESS_MEM_TYPE_64 |
+                                          PCI_BASE_ADDRESS_MEM_PREFETCH);
        }
 }
 
diff --git a/drivers/pci/endpoint/pci-epf-core.c b/drivers/pci/endpoint/pci-epf-core.c
index 0a28a0b0911b..acb93055181b 100644
--- a/drivers/pci/endpoint/pci-epf-core.c
+++ b/drivers/pci/endpoint/pci-epf-core.c
@@ -305,7 +305,8 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar,
        epf_bar[bar].size = size;
        epf_bar[bar].barno = bar;
        epf_bar[bar].flags |= upper_32_bits(size) ?
-                               PCI_BASE_ADDRESS_MEM_TYPE_64 :
+                               (PCI_BASE_ADDRESS_MEM_TYPE_64 |
+                                PCI_BASE_ADDRESS_MEM_PREFETCH) :
                                PCI_BASE_ADDRESS_MEM_TYPE_32;
 
        return space;


Now when I look at it, the whole "if (epc_features->bar[i].only_64bit)"
should move to pci_epf_alloc_space() IMO, so that not all EPF drivers need to
duplicate this code.


Kind regards,
Niklas