* [PATCH v2 0/2] PCI: dwc: Add support for 64-bit MSI target addresses @ 2022-08-10 18:35 Will McVicker 2022-08-10 18:35 ` [PATCH v2 1/2] PCI: dwc: drop dependency on ZONE_DMA32 Will McVicker 2022-08-10 18:35 ` [PATCH v2 2/2] PCI: dwc: add support for 64-bit MSI target address Will McVicker 0 siblings, 2 replies; 6+ messages in thread From: Will McVicker @ 2022-08-10 18:35 UTC (permalink / raw) To: Jingoo Han, Gustavo Pimentel, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Bjorn Helgaas, Will McVicker Cc: kernel-team, Vidya Sagar, kernel test robot, Isaac J . Manjarres, linux-pci, linux-kernel Hi All, I have updated the patch series based on the build error reported by the kernel test robot which found an issues with the new inline function when PCIE_DW_HOST isn't set. Additionally, I have updated the DMA alloc error handling for the first patch which was reported to me directly by Isaac Manjarres. Basically, we don't need to check for a DMA mapping error since dma_alloc_coherent() will free the memory when a mapping error occurs. Please take a look and let me know if you have any questions or concerns. Thanks, Will Reported-by: kernel test robot <lkp@intel.com> Reported-by: Isaac J. Manjarres <isaacmanjarres@google.com> Will McVicker (2): PCI: dwc: drop dependency on ZONE_DMA32 PCI: dwc: add support for 64-bit MSI target address v2: * Fixed build error caught by kernel test robot * Fixed error handling reported by Isaac Manjarres .../pci/controller/dwc/pcie-designware-host.c | 37 +++++++++++-------- drivers/pci/controller/dwc/pcie-designware.c | 9 +++++ drivers/pci/controller/dwc/pcie-designware.h | 1 + 3 files changed, 31 insertions(+), 16 deletions(-) base-commit: d4252071b97d2027d246f6a82cbee4d52f618b47 -- 2.37.1.559.g78731f0fdb-goog ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 1/2] PCI: dwc: drop dependency on ZONE_DMA32 2022-08-10 18:35 [PATCH v2 0/2] PCI: dwc: Add support for 64-bit MSI target addresses Will McVicker @ 2022-08-10 18:35 ` Will McVicker 2022-08-10 21:27 ` Rob Herring 2022-08-10 18:35 ` [PATCH v2 2/2] PCI: dwc: add support for 64-bit MSI target address Will McVicker 1 sibling, 1 reply; 6+ messages in thread From: Will McVicker @ 2022-08-10 18:35 UTC (permalink / raw) To: Jingoo Han, Gustavo Pimentel, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Bjorn Helgaas, Will McVicker Cc: kernel-team, Vidya Sagar, kernel test robot, Isaac J . Manjarres, linux-pci, linux-kernel Re-work the msi_msg DMA allocation logic to use dma_alloc_coherent() which uses the coherent DMA mask to try and return an allocation within the DMA mask limits. This allows kernel configurations that disable ZONE_DMA32 to continue supporting a 32-bit DMA mask. Without this patch, the PCIe host device will fail to probe when ZONE_DMA32 is disabled. Fixes: 35797e672ff0 ("PCI: dwc: Fix MSI msi_msg DMA mapping") Reported-by: Isaac J. Manjarres <isaacmanjarres@google.com> Signed-off-by: Will McVicker <willmcvicker@google.com> --- .../pci/controller/dwc/pcie-designware-host.c | 23 ++++++++----------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index 7746f94a715f..8f2222f51671 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -272,9 +272,9 @@ static void dw_pcie_free_msi(struct dw_pcie_rp *pp) struct dw_pcie *pci = to_dw_pcie_from_pp(pp); struct device *dev = pci->dev; - dma_unmap_page(dev, pp->msi_data, PAGE_SIZE, DMA_FROM_DEVICE); - if (pp->msi_page) - __free_page(pp->msi_page); + dma_free_coherent(dev, PAGE_SIZE, pp->msi_page, pp->msi_data); + pp->msi_data = 0; + pp->msi_page = NULL; } } @@ -375,22 +375,17 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) dw_chained_msi_isr, pp); } - ret = dma_set_mask(dev, DMA_BIT_MASK(32)); + ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); if (ret) dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n"); - pp->msi_page = alloc_page(GFP_DMA32); - pp->msi_data = dma_map_page(dev, pp->msi_page, 0, - PAGE_SIZE, DMA_FROM_DEVICE); - ret = dma_mapping_error(dev, pp->msi_data); - if (ret) { - dev_err(pci->dev, "Failed to map MSI data\n"); - __free_page(pp->msi_page); - pp->msi_page = NULL; + pp->msi_page = dma_alloc_coherent(dev, PAGE_SIZE, &pp->msi_data, + GFP_KERNEL); + if (!pp->msi_page) { + dev_err(dev, "Failed to alloc and map MSI data\n"); pp->msi_data = 0; dw_pcie_free_msi(pp); - - return ret; + return -ENOMEM; } return 0; -- 2.37.1.559.g78731f0fdb-goog ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 1/2] PCI: dwc: drop dependency on ZONE_DMA32 2022-08-10 18:35 ` [PATCH v2 1/2] PCI: dwc: drop dependency on ZONE_DMA32 Will McVicker @ 2022-08-10 21:27 ` Rob Herring 2022-08-10 23:09 ` William McVicker 0 siblings, 1 reply; 6+ messages in thread From: Rob Herring @ 2022-08-10 21:27 UTC (permalink / raw) To: Will McVicker Cc: Jingoo Han, Gustavo Pimentel, Lorenzo Pieralisi, Krzysztof Wilczyński, Bjorn Helgaas, kernel-team, Vidya Sagar, kernel test robot, Isaac J . Manjarres, linux-pci, linux-kernel On Wed, Aug 10, 2022 at 06:35:34PM +0000, Will McVicker wrote: > Re-work the msi_msg DMA allocation logic to use dma_alloc_coherent() > which uses the coherent DMA mask to try and return an allocation within > the DMA mask limits. This allows kernel configurations that disable > ZONE_DMA32 to continue supporting a 32-bit DMA mask. Without this patch, > the PCIe host device will fail to probe when ZONE_DMA32 is disabled. > > Fixes: 35797e672ff0 ("PCI: dwc: Fix MSI msi_msg DMA mapping") > Reported-by: Isaac J. Manjarres <isaacmanjarres@google.com> > Signed-off-by: Will McVicker <willmcvicker@google.com> > --- > .../pci/controller/dwc/pcie-designware-host.c | 23 ++++++++----------- > 1 file changed, 9 insertions(+), 14 deletions(-) > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c > index 7746f94a715f..8f2222f51671 100644 > --- a/drivers/pci/controller/dwc/pcie-designware-host.c > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c > @@ -272,9 +272,9 @@ static void dw_pcie_free_msi(struct dw_pcie_rp *pp) > struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > struct device *dev = pci->dev; > > - dma_unmap_page(dev, pp->msi_data, PAGE_SIZE, DMA_FROM_DEVICE); > - if (pp->msi_page) > - __free_page(pp->msi_page); > + dma_free_coherent(dev, PAGE_SIZE, pp->msi_page, pp->msi_data); > + pp->msi_data = 0; > + pp->msi_page = NULL; > } > } > > @@ -375,22 +375,17 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) > dw_chained_msi_isr, pp); > } > > - ret = dma_set_mask(dev, DMA_BIT_MASK(32)); > + ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); > if (ret) > dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n"); > > - pp->msi_page = alloc_page(GFP_DMA32); > - pp->msi_data = dma_map_page(dev, pp->msi_page, 0, > - PAGE_SIZE, DMA_FROM_DEVICE); > - ret = dma_mapping_error(dev, pp->msi_data); > - if (ret) { > - dev_err(pci->dev, "Failed to map MSI data\n"); > - __free_page(pp->msi_page); > - pp->msi_page = NULL; > + pp->msi_page = dma_alloc_coherent(dev, PAGE_SIZE, &pp->msi_data, > + GFP_KERNEL); You can use the managed version, dmam_alloc_coherent(), and avoid the freeing yourself. Also with that, I think you don't need 'msi_page'? Also, no need to alloc a whole page. A u32 or u64? should be fine. The write never makes it to memory, so doesn't really matter. > + if (!pp->msi_page) { > + dev_err(dev, "Failed to alloc and map MSI data\n"); > pp->msi_data = 0; > dw_pcie_free_msi(pp); > - > - return ret; > + return -ENOMEM; > } > > return 0; > -- > 2.37.1.559.g78731f0fdb-goog > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 1/2] PCI: dwc: drop dependency on ZONE_DMA32 2022-08-10 21:27 ` Rob Herring @ 2022-08-10 23:09 ` William McVicker 0 siblings, 0 replies; 6+ messages in thread From: William McVicker @ 2022-08-10 23:09 UTC (permalink / raw) To: Rob Herring Cc: Jingoo Han, Gustavo Pimentel, Lorenzo Pieralisi, Krzysztof Wilczyński, Bjorn Helgaas, kernel-team, Vidya Sagar, kernel test robot, Isaac J . Manjarres, linux-pci, linux-kernel On 08/10/2022, Rob Herring wrote: > On Wed, Aug 10, 2022 at 06:35:34PM +0000, Will McVicker wrote: > > Re-work the msi_msg DMA allocation logic to use dma_alloc_coherent() > > which uses the coherent DMA mask to try and return an allocation within > > the DMA mask limits. This allows kernel configurations that disable > > ZONE_DMA32 to continue supporting a 32-bit DMA mask. Without this patch, > > the PCIe host device will fail to probe when ZONE_DMA32 is disabled. > > > > Fixes: 35797e672ff0 ("PCI: dwc: Fix MSI msi_msg DMA mapping") > > Reported-by: Isaac J. Manjarres <isaacmanjarres@google.com> > > Signed-off-by: Will McVicker <willmcvicker@google.com> > > --- > > .../pci/controller/dwc/pcie-designware-host.c | 23 ++++++++----------- > > 1 file changed, 9 insertions(+), 14 deletions(-) > > > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c > > index 7746f94a715f..8f2222f51671 100644 > > --- a/drivers/pci/controller/dwc/pcie-designware-host.c > > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c > > @@ -272,9 +272,9 @@ static void dw_pcie_free_msi(struct dw_pcie_rp *pp) > > struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > > struct device *dev = pci->dev; > > > > - dma_unmap_page(dev, pp->msi_data, PAGE_SIZE, DMA_FROM_DEVICE); > > - if (pp->msi_page) > > - __free_page(pp->msi_page); > > + dma_free_coherent(dev, PAGE_SIZE, pp->msi_page, pp->msi_data); > > + pp->msi_data = 0; > > + pp->msi_page = NULL; > > } > > } > > > > @@ -375,22 +375,17 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) > > dw_chained_msi_isr, pp); > > } > > > > - ret = dma_set_mask(dev, DMA_BIT_MASK(32)); > > + ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); > > if (ret) > > dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n"); > > > > - pp->msi_page = alloc_page(GFP_DMA32); > > - pp->msi_data = dma_map_page(dev, pp->msi_page, 0, > > - PAGE_SIZE, DMA_FROM_DEVICE); > > - ret = dma_mapping_error(dev, pp->msi_data); > > - if (ret) { > > - dev_err(pci->dev, "Failed to map MSI data\n"); > > - __free_page(pp->msi_page); > > - pp->msi_page = NULL; > > + pp->msi_page = dma_alloc_coherent(dev, PAGE_SIZE, &pp->msi_data, > > + GFP_KERNEL); > > You can use the managed version, dmam_alloc_coherent(), and avoid the > freeing yourself. Also with that, I think you don't need 'msi_page'? > > Also, no need to alloc a whole page. A u32 or u64? should be fine. The > write never makes it to memory, so doesn't really matter. > > > + if (!pp->msi_page) { > > + dev_err(dev, "Failed to alloc and map MSI data\n"); > > pp->msi_data = 0; > > dw_pcie_free_msi(pp); > > - > > - return ret; > > + return -ENOMEM; > > } > > > > return 0; > > -- > > 2.37.1.559.g78731f0fdb-goog > > Thanks for the review Rob! I've updated the series to address your concerns. Regards, Will ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 2/2] PCI: dwc: add support for 64-bit MSI target address 2022-08-10 18:35 [PATCH v2 0/2] PCI: dwc: Add support for 64-bit MSI target addresses Will McVicker 2022-08-10 18:35 ` [PATCH v2 1/2] PCI: dwc: drop dependency on ZONE_DMA32 Will McVicker @ 2022-08-10 18:35 ` Will McVicker 2022-08-10 21:30 ` Rob Herring 1 sibling, 1 reply; 6+ messages in thread From: Will McVicker @ 2022-08-10 18:35 UTC (permalink / raw) To: Jingoo Han, Gustavo Pimentel, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Bjorn Helgaas, Will McVicker Cc: kernel-team, Vidya Sagar, kernel test robot, Isaac J . Manjarres, linux-pci, linux-kernel Since not all devices require a 32-bit MSI address, add support to the PCIe host driver to allow setting the DMA mask to 64-bits. This allows kernels to disable ZONE_DMA32 and bounce buffering (swiotlb) without risking not being able to get a 32-bit address during DMA allocation. Basically, in the slim chance that there are no 32-bit allocations available, the current PCIe host driver will fail to allocate the msi_msg page due to a DMA address overflow (seen in [1]). With this patch, the PCIe driver can advertise 64-bit support via it's MSI capabilities to hint to the PCIe host driver to set the DMA mask to 64-bits. [1] https://lore.kernel.org/all/Yo0soniFborDl7+C@google.com/ Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Will McVicker <willmcvicker@google.com> --- drivers/pci/controller/dwc/pcie-designware-host.c | 14 ++++++++++++-- drivers/pci/controller/dwc/pcie-designware.c | 9 +++++++++ drivers/pci/controller/dwc/pcie-designware.h | 1 + 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index 8f2222f51671..2b9b63489415 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -338,6 +338,8 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) struct platform_device *pdev = to_platform_device(dev); int ret; u32 ctrl, num_ctrls; + bool msi_64b = false; + u16 msi_capabilities; for (ctrl = 0; ctrl < MAX_MSI_CTRLS; ctrl++) pp->irq_mask[ctrl] = ~0; @@ -375,9 +377,17 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) dw_chained_msi_isr, pp); } - ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); + msi_capabilities = dw_pcie_msi_capabilities(pci); + if (msi_capabilities & PCI_MSI_FLAGS_ENABLE) + msi_64b = msi_capabilities & PCI_MSI_FLAGS_64BIT ? true : false; + + dev_dbg(dev, "Setting MSI DMA mask to %s-bit.\n", + msi_64b ? "64" : "32"); + ret = dma_set_mask_and_coherent(dev, msi_64b ? + DMA_BIT_MASK(64) : DMA_BIT_MASK(32)); if (ret) - dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n"); + dev_warn(dev, "Failed to set DMA mask to %s-bit.\n", + msi_64b ? "64" : "32"); pp->msi_page = dma_alloc_coherent(dev, PAGE_SIZE, &pp->msi_data, GFP_KERNEL); diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index c6725c519a47..8ed402307d7f 100644 --- a/drivers/pci/controller/dwc/pcie-designware.c +++ b/drivers/pci/controller/dwc/pcie-designware.c @@ -82,6 +82,15 @@ u8 dw_pcie_find_capability(struct dw_pcie *pci, u8 cap) } EXPORT_SYMBOL_GPL(dw_pcie_find_capability); +u16 dw_pcie_msi_capabilities(struct dw_pcie *pci) +{ + u8 offset; + + offset = dw_pcie_find_capability(pci, PCI_CAP_ID_MSI); + return dw_pcie_readw_dbi(pci, offset + PCI_MSI_FLAGS); +} +EXPORT_SYMBOL_GPL(dw_pcie_msi_capabilities); + static u16 dw_pcie_find_next_ext_capability(struct dw_pcie *pci, u16 start, u8 cap) { diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 09b887093a84..4631e26ba6c5 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -333,6 +333,7 @@ void dw_pcie_version_detect(struct dw_pcie *pci); u8 dw_pcie_find_capability(struct dw_pcie *pci, u8 cap); u16 dw_pcie_find_ext_capability(struct dw_pcie *pci, u8 cap); +u16 dw_pcie_msi_capabilities(struct dw_pcie *pci); int dw_pcie_read(void __iomem *addr, int size, u32 *val); int dw_pcie_write(void __iomem *addr, int size, u32 val); -- 2.37.1.559.g78731f0fdb-goog ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] PCI: dwc: add support for 64-bit MSI target address 2022-08-10 18:35 ` [PATCH v2 2/2] PCI: dwc: add support for 64-bit MSI target address Will McVicker @ 2022-08-10 21:30 ` Rob Herring 0 siblings, 0 replies; 6+ messages in thread From: Rob Herring @ 2022-08-10 21:30 UTC (permalink / raw) To: Will McVicker Cc: Jingoo Han, Gustavo Pimentel, Lorenzo Pieralisi, Krzysztof Wilczyński, Bjorn Helgaas, kernel-team, Vidya Sagar, kernel test robot, Isaac J . Manjarres, linux-pci, linux-kernel On Wed, Aug 10, 2022 at 06:35:35PM +0000, Will McVicker wrote: > Since not all devices require a 32-bit MSI address, add support to the > PCIe host driver to allow setting the DMA mask to 64-bits. This allows > kernels to disable ZONE_DMA32 and bounce buffering (swiotlb) without > risking not being able to get a 32-bit address during DMA allocation. > Basically, in the slim chance that there are no 32-bit allocations > available, the current PCIe host driver will fail to allocate the > msi_msg page due to a DMA address overflow (seen in [1]). With this > patch, the PCIe driver can advertise 64-bit support via it's MSI > capabilities to hint to the PCIe host driver to set the DMA mask to > 64-bits. > > [1] https://lore.kernel.org/all/Yo0soniFborDl7+C@google.com/ > > Reported-by: kernel test robot <lkp@intel.com> > Signed-off-by: Will McVicker <willmcvicker@google.com> > --- > drivers/pci/controller/dwc/pcie-designware-host.c | 14 ++++++++++++-- > drivers/pci/controller/dwc/pcie-designware.c | 9 +++++++++ > drivers/pci/controller/dwc/pcie-designware.h | 1 + > 3 files changed, 22 insertions(+), 2 deletions(-) Reviewed-by: Rob Herring <robh@kernel.org> ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-08-10 23:09 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-08-10 18:35 [PATCH v2 0/2] PCI: dwc: Add support for 64-bit MSI target addresses Will McVicker 2022-08-10 18:35 ` [PATCH v2 1/2] PCI: dwc: drop dependency on ZONE_DMA32 Will McVicker 2022-08-10 21:27 ` Rob Herring 2022-08-10 23:09 ` William McVicker 2022-08-10 18:35 ` [PATCH v2 2/2] PCI: dwc: add support for 64-bit MSI target address Will McVicker 2022-08-10 21:30 ` Rob Herring
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).