* [PATCH 0/2] xen: swiotlb: 2 fixes SoCs w/o IOMMU (e.g. iMX8QXP) @ 2025-05-02 11:40 John Ernberg 2025-05-02 11:40 ` [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it John Ernberg 2025-05-02 11:40 ` [PATCH 2/2] xen: swiotlb: Implement map_resource callback John Ernberg 0 siblings, 2 replies; 11+ messages in thread From: John Ernberg @ 2025-05-02 11:40 UTC (permalink / raw) To: Juergen Gross, Stefano Stabellini Cc: Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, John Ernberg There's 2 problems with DMA today when running Xen on the iMX8QXP SoC. The first identifies as a USB corruption, but is actually a memory corruption risk with any small DMA transfer, and just manifests itself in USB transfers. This is a regression fix tracing back to Linux 6.5 when the blamed commit (patch series) landed. The second one causes any attempt to use DMA via the iMX8QXP eDMA v3 block to fail with DMA_MAPPING_ERROR when running under Xen because this DMA controller wants to do DMA in MMIO space. I'm a little bit more on the fence with the second one, as that never worked, but is still fixing an issue. There is no Fixes or Cc stable on this one because of this. John Ernberg (2): xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it xen: swiotlb: Implement map_resource callback drivers/xen/swiotlb-xen.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) -- 2.49.0 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it 2025-05-02 11:40 [PATCH 0/2] xen: swiotlb: 2 fixes SoCs w/o IOMMU (e.g. iMX8QXP) John Ernberg @ 2025-05-02 11:40 ` John Ernberg 2025-05-02 17:07 ` Stefano Stabellini 2025-05-08 15:21 ` Catalin Marinas 2025-05-02 11:40 ` [PATCH 2/2] xen: swiotlb: Implement map_resource callback John Ernberg 1 sibling, 2 replies; 11+ messages in thread From: John Ernberg @ 2025-05-02 11:40 UTC (permalink / raw) To: Juergen Gross, Stefano Stabellini Cc: Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, John Ernberg, stable@kernel.org Xen swiotlb support was missed when the patch set starting with 4ab5f8ec7d71 ("mm/slab: decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN") was merged. When running Xen on iMX8QXP, a SoC without IOMMU, the effect was that USB transfers ended up corrupted when there was more than one URB inflight at the same time. Add a call to dma_kmalloc_needs_bounce() to make sure that allocations too small for DMA get bounced via swiotlb. Closes: https://lore.kernel.org/linux-usb/ab2776f0-b838-4cf6-a12a-c208eb6aad59@actia.se/ Fixes: 4ab5f8ec7d71 ("mm/slab: decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN") Cc: stable@kernel.org # v6.5+ Signed-off-by: John Ernberg <john.ernberg@actia.se> --- It's impossible to pick an exact fixes tag since this driver was missed in the flagged patch set. I picked one I felt gave a decent enough picture for someone coming across this later. --- drivers/xen/swiotlb-xen.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c index 1f65795cf5d7..ef56a2500ed6 100644 --- a/drivers/xen/swiotlb-xen.c +++ b/drivers/xen/swiotlb-xen.c @@ -217,6 +217,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, * buffering it. */ if (dma_capable(dev, dev_addr, size, true) && + !dma_kmalloc_needs_bounce(dev, size, dir) && !range_straddles_page_boundary(phys, size) && !xen_arch_need_swiotlb(dev, phys, dev_addr) && !is_swiotlb_force_bounce(dev)) -- 2.49.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it 2025-05-02 11:40 ` [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it John Ernberg @ 2025-05-02 17:07 ` Stefano Stabellini 2025-05-08 15:21 ` Catalin Marinas 1 sibling, 0 replies; 11+ messages in thread From: Stefano Stabellini @ 2025-05-02 17:07 UTC (permalink / raw) To: John Ernberg Cc: Juergen Gross, Stefano Stabellini, Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, stable@kernel.org On Fri, 2 May 2025, John Ernberg wrote: > Xen swiotlb support was missed when the patch set starting with > 4ab5f8ec7d71 ("mm/slab: decouple ARCH_KMALLOC_MINALIGN from > ARCH_DMA_MINALIGN") was merged. > > When running Xen on iMX8QXP, a SoC without IOMMU, the effect was that USB > transfers ended up corrupted when there was more than one URB inflight at > the same time. > > Add a call to dma_kmalloc_needs_bounce() to make sure that allocations too > small for DMA get bounced via swiotlb. > > Closes: https://lore.kernel.org/linux-usb/ab2776f0-b838-4cf6-a12a-c208eb6aad59@actia.se/ > Fixes: 4ab5f8ec7d71 ("mm/slab: decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN") > Cc: stable@kernel.org # v6.5+ > Signed-off-by: John Ernberg <john.ernberg@actia.se> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org> > --- > > It's impossible to pick an exact fixes tag since this driver was missed > in the flagged patch set. I picked one I felt gave a decent enough picture > for someone coming across this later. > --- > drivers/xen/swiotlb-xen.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c > index 1f65795cf5d7..ef56a2500ed6 100644 > --- a/drivers/xen/swiotlb-xen.c > +++ b/drivers/xen/swiotlb-xen.c > @@ -217,6 +217,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, > * buffering it. > */ > if (dma_capable(dev, dev_addr, size, true) && > + !dma_kmalloc_needs_bounce(dev, size, dir) && > !range_straddles_page_boundary(phys, size) && > !xen_arch_need_swiotlb(dev, phys, dev_addr) && > !is_swiotlb_force_bounce(dev)) > -- > 2.49.0 > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it 2025-05-02 11:40 ` [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it John Ernberg 2025-05-02 17:07 ` Stefano Stabellini @ 2025-05-08 15:21 ` Catalin Marinas 1 sibling, 0 replies; 11+ messages in thread From: Catalin Marinas @ 2025-05-08 15:21 UTC (permalink / raw) To: John Ernberg Cc: Juergen Gross, Stefano Stabellini, Oleksandr Tyshchenko, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, stable@kernel.org On Fri, May 02, 2025 at 11:40:55AM +0000, John Ernberg wrote: > Xen swiotlb support was missed when the patch set starting with > 4ab5f8ec7d71 ("mm/slab: decouple ARCH_KMALLOC_MINALIGN from > ARCH_DMA_MINALIGN") was merged. > > When running Xen on iMX8QXP, a SoC without IOMMU, the effect was that USB > transfers ended up corrupted when there was more than one URB inflight at > the same time. > > Add a call to dma_kmalloc_needs_bounce() to make sure that allocations too > small for DMA get bounced via swiotlb. > > Closes: https://lore.kernel.org/linux-usb/ab2776f0-b838-4cf6-a12a-c208eb6aad59@actia.se/ > Fixes: 4ab5f8ec7d71 ("mm/slab: decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN") > Cc: stable@kernel.org # v6.5+ > Signed-off-by: John Ernberg <john.ernberg@actia.se> > > --- > > It's impossible to pick an exact fixes tag since this driver was missed > in the flagged patch set. I picked one I felt gave a decent enough picture > for someone coming across this later. All the above patches went in at the same time in 6.5, so it probably doesn't matter. In theory, you could add: Fixes: 370645f41e6e ("dma-mapping: force bouncing if the kmalloc() size is not cache-line-aligned") Cc: <stable@vger.kernel.org> # 6.5.x as that's when dma_kmalloc_needs_bounce() was added (a few commits after the "decouple ARCH_KMALLOC_MINALIGN..." one). However, actual problems started to appear with commit 9382bc44b5f5 ("arm64: allow kmalloc() caches aligned to the smaller cache_line_size()") which makes ARCH_KMALLOC_MINALIGN equal 8 on arm64. > --- > drivers/xen/swiotlb-xen.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c > index 1f65795cf5d7..ef56a2500ed6 100644 > --- a/drivers/xen/swiotlb-xen.c > +++ b/drivers/xen/swiotlb-xen.c > @@ -217,6 +217,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, > * buffering it. > */ > if (dma_capable(dev, dev_addr, size, true) && > + !dma_kmalloc_needs_bounce(dev, size, dir) && > !range_straddles_page_boundary(phys, size) && > !xen_arch_need_swiotlb(dev, phys, dev_addr) && > !is_swiotlb_force_bounce(dev)) Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 2/2] xen: swiotlb: Implement map_resource callback 2025-05-02 11:40 [PATCH 0/2] xen: swiotlb: 2 fixes SoCs w/o IOMMU (e.g. iMX8QXP) John Ernberg 2025-05-02 11:40 ` [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it John Ernberg @ 2025-05-02 11:40 ` John Ernberg 2025-05-02 17:20 ` Stefano Stabellini 1 sibling, 1 reply; 11+ messages in thread From: John Ernberg @ 2025-05-02 11:40 UTC (permalink / raw) To: Juergen Gross, Stefano Stabellini Cc: Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, John Ernberg Needed by the eDMA v3 DMA engine found in iommu-less SoCs such as iMX8QXP to be able to perform DMA operations as a Xen Hardware Domain, which needs to be able to do DMA in MMIO space. The callback implementation is basically the same as the one for direct mapping of resources, except this also takes into account Xen page mappings. There is nothing to do for unmap, just like with direct, so the unmap callback is not needed. Signed-off-by: John Ernberg <john.ernberg@actia.se> --- I originally exported dma_direct_map_resource() and used that which appeared to work, but it felt like not checking Xen page mappings wasn't fully correct and I went with this. But if dma_direct_map_resource() is the correct approach here then I can submit that isntead. --- drivers/xen/swiotlb-xen.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c index ef56a2500ed6..0f02fdd9128d 100644 --- a/drivers/xen/swiotlb-xen.c +++ b/drivers/xen/swiotlb-xen.c @@ -285,6 +285,20 @@ static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr, attrs, pool); } +static dma_addr_t xen_swiotlb_map_resource(struct device *dev, phys_addr_t phys, + size_t size, enum dma_data_direction dir, + unsigned long attrs) +{ + dma_addr_t dev_addr = xen_phys_to_dma(dev, phys); + + BUG_ON(dir == DMA_NONE); + + if (!dma_capable(dev, dev_addr, size, false)) + return DMA_MAPPING_ERROR; + + return dev_addr; +} + static void xen_swiotlb_sync_single_for_cpu(struct device *dev, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) @@ -426,4 +440,5 @@ const struct dma_map_ops xen_swiotlb_dma_ops = { .alloc_pages_op = dma_common_alloc_pages, .free_pages = dma_common_free_pages, .max_mapping_size = swiotlb_max_mapping_size, + .map_resource = xen_swiotlb_map_resource, }; -- 2.49.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] xen: swiotlb: Implement map_resource callback 2025-05-02 11:40 ` [PATCH 2/2] xen: swiotlb: Implement map_resource callback John Ernberg @ 2025-05-02 17:20 ` Stefano Stabellini 2025-05-06 12:22 ` John Ernberg 0 siblings, 1 reply; 11+ messages in thread From: Stefano Stabellini @ 2025-05-02 17:20 UTC (permalink / raw) To: John Ernberg Cc: Juergen Gross, Stefano Stabellini, Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, Christoph Hellwig +Christoph On Fri, 2 May 2025, John Ernberg wrote: > Needed by the eDMA v3 DMA engine found in iommu-less SoCs such as iMX8QXP > to be able to perform DMA operations as a Xen Hardware Domain, which needs > to be able to do DMA in MMIO space. > > The callback implementation is basically the same as the one for direct > mapping of resources, except this also takes into account Xen page > mappings. > > There is nothing to do for unmap, just like with direct, so the unmap > callback is not needed. > > Signed-off-by: John Ernberg <john.ernberg@actia.se> > > --- > > I originally exported dma_direct_map_resource() and used that which > appeared to work, but it felt like not checking Xen page mappings wasn't > fully correct and I went with this. But if dma_direct_map_resource() is > the correct approach here then I can submit that isntead. > --- > drivers/xen/swiotlb-xen.c | 15 +++++++++++++++ > 1 file changed, 15 insertions(+) > > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c > index ef56a2500ed6..0f02fdd9128d 100644 > --- a/drivers/xen/swiotlb-xen.c > +++ b/drivers/xen/swiotlb-xen.c > @@ -285,6 +285,20 @@ static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr, > attrs, pool); > } > > +static dma_addr_t xen_swiotlb_map_resource(struct device *dev, phys_addr_t phys, > + size_t size, enum dma_data_direction dir, > + unsigned long attrs) > +{ > + dma_addr_t dev_addr = xen_phys_to_dma(dev, phys); Yes, we need the xen_phys_to_dma call here. This is one of the reasons I don't think we can use dma_direct_map_resource() to implement map_resource > + BUG_ON(dir == DMA_NONE); > + > + if (!dma_capable(dev, dev_addr, size, false)) > + return DMA_MAPPING_ERROR; But here you also need to check that phys+size doesn't cross a page boundary. You need to call range_straddles_page_boundary, like we do at the beginning of xen_swiotlb_map_page. If it is possible to cross a page boundary, then we need to basically to do the same thing here as we do in xen_swiotlb_map_page where we check for: if (dma_capable(dev, dev_addr, size, true) && !range_straddles_page_boundary(phys, size) && !xen_arch_need_swiotlb(dev, phys, dev_addr) && !is_swiotlb_force_bounce(dev)) goto done; if all is well, we go with the native path, otherwise we bounce on swiotlb-xen. > + return dev_addr; > +} > + > static void > xen_swiotlb_sync_single_for_cpu(struct device *dev, dma_addr_t dma_addr, > size_t size, enum dma_data_direction dir) > @@ -426,4 +440,5 @@ const struct dma_map_ops xen_swiotlb_dma_ops = { > .alloc_pages_op = dma_common_alloc_pages, > .free_pages = dma_common_free_pages, > .max_mapping_size = swiotlb_max_mapping_size, > + .map_resource = xen_swiotlb_map_resource, > }; > -- > 2.49.0 > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] xen: swiotlb: Implement map_resource callback 2025-05-02 17:20 ` Stefano Stabellini @ 2025-05-06 12:22 ` John Ernberg 2025-05-07 23:09 ` Stefano Stabellini 0 siblings, 1 reply; 11+ messages in thread From: John Ernberg @ 2025-05-06 12:22 UTC (permalink / raw) To: Stefano Stabellini Cc: Juergen Gross, Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, Christoph Hellwig Hi Stefano, On 5/2/25 7:20 PM, Stefano Stabellini wrote: > +Christoph > > On Fri, 2 May 2025, John Ernberg wrote: >> Needed by the eDMA v3 DMA engine found in iommu-less SoCs such as iMX8QXP >> to be able to perform DMA operations as a Xen Hardware Domain, which needs >> to be able to do DMA in MMIO space. Self-note: The above part of the commit message is a disaster and should read something like. Needed by SoCs without an iommu (such as the iMX8QXP and it's eDMA v3 engine) that need to be able to perform DMA operations in MMIO space. >> >> The callback implementation is basically the same as the one for direct >> mapping of resources, except this also takes into account Xen page >> mappings. >> >> There is nothing to do for unmap, just like with direct, so the unmap >> callback is not needed. >> >> Signed-off-by: John Ernberg <john.ernberg@actia.se> >> >> --- >> >> I originally exported dma_direct_map_resource() and used that which >> appeared to work, but it felt like not checking Xen page mappings wasn't >> fully correct and I went with this. But if dma_direct_map_resource() is >> the correct approach here then I can submit that isntead. >> --- >> drivers/xen/swiotlb-xen.c | 15 +++++++++++++++ >> 1 file changed, 15 insertions(+) >> >> diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c >> index ef56a2500ed6..0f02fdd9128d 100644 >> --- a/drivers/xen/swiotlb-xen.c >> +++ b/drivers/xen/swiotlb-xen.c >> @@ -285,6 +285,20 @@ static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr, >> attrs, pool); >> } >> >> +static dma_addr_t xen_swiotlb_map_resource(struct device *dev, phys_addr_t phys, >> + size_t size, enum dma_data_direction dir, >> + unsigned long attrs) >> +{ >> + dma_addr_t dev_addr = xen_phys_to_dma(dev, phys); > > Yes, we need the xen_phys_to_dma call here. This is one of the reasons I > don't think we can use dma_direct_map_resource() to implement > map_resource > > >> + BUG_ON(dir == DMA_NONE); >> + >> + if (!dma_capable(dev, dev_addr, size, false)) >> + return DMA_MAPPING_ERROR; > > But here you also need to check that phys+size doesn't cross a page > boundary. You need to call range_straddles_page_boundary, like we do at > the beginning of xen_swiotlb_map_page. > > If it is possible to cross a page boundary, then we need to basically to > do the same thing here as we do in xen_swiotlb_map_page where we check > for: > > if (dma_capable(dev, dev_addr, size, true) && > !range_straddles_page_boundary(phys, size) && > !xen_arch_need_swiotlb(dev, phys, dev_addr) && > !is_swiotlb_force_bounce(dev)) > goto done; > > if all is well, we go with the native path, otherwise we bounce on > swiotlb-xen. > I'll preface this with that it's my first deep dive in DMA, so the following may entirely be me being stupid: I'm not sure a straddles page boundary path makes sense. This mapping is not for a RAM backed address. In the eDMA case for the iMX8QXP the `phys` coming in here is the address of a register. I cannot see how a swiotlb bounce would fix anything if you end up straddling a page boundary. At most I can see a WARN_ON with a DMA_MAPPING_ERROR return if such a check would yield true. Let's say you want to do a DEV_TO_MEM and a check decides you need to bounce it you'd bounce an RX register address. I cannot see how that DMA would ever end up doing the expected thing. The eDMA engine will manage the RX/TX registers for an IP block and move the data between them and RAM, where the RAM memory is mapped separately by dma_map_page (and family). And these can be swiotlb bounced via the map page callback with no problem. Best regards // John Ernberg ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] xen: swiotlb: Implement map_resource callback 2025-05-06 12:22 ` John Ernberg @ 2025-05-07 23:09 ` Stefano Stabellini 2025-05-08 4:14 ` Christoph Hellwig 0 siblings, 1 reply; 11+ messages in thread From: Stefano Stabellini @ 2025-05-07 23:09 UTC (permalink / raw) To: John Ernberg Cc: Stefano Stabellini, Juergen Gross, Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, Christoph Hellwig On Tue, 6 May 2025, John Ernberg wrote: > Hi Stefano, > > On 5/2/25 7:20 PM, Stefano Stabellini wrote: > > +Christoph > > > > On Fri, 2 May 2025, John Ernberg wrote: > >> Needed by the eDMA v3 DMA engine found in iommu-less SoCs such as iMX8QXP > >> to be able to perform DMA operations as a Xen Hardware Domain, which needs > >> to be able to do DMA in MMIO space. > > Self-note: The above part of the commit message is a disaster and should > read something like. > > Needed by SoCs without an iommu (such as the iMX8QXP and it's eDMA v3 > engine) that need to be able to perform DMA operations in MMIO space. > > >> > >> The callback implementation is basically the same as the one for direct > >> mapping of resources, except this also takes into account Xen page > >> mappings. > >> > >> There is nothing to do for unmap, just like with direct, so the unmap > >> callback is not needed. > >> > >> Signed-off-by: John Ernberg <john.ernberg@actia.se> > >> > >> --- > >> > >> I originally exported dma_direct_map_resource() and used that which > >> appeared to work, but it felt like not checking Xen page mappings wasn't > >> fully correct and I went with this. But if dma_direct_map_resource() is > >> the correct approach here then I can submit that isntead. > >> --- > >> drivers/xen/swiotlb-xen.c | 15 +++++++++++++++ > >> 1 file changed, 15 insertions(+) > >> > >> diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c > >> index ef56a2500ed6..0f02fdd9128d 100644 > >> --- a/drivers/xen/swiotlb-xen.c > >> +++ b/drivers/xen/swiotlb-xen.c > >> @@ -285,6 +285,20 @@ static void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr, > >> attrs, pool); > >> } > >> > >> +static dma_addr_t xen_swiotlb_map_resource(struct device *dev, phys_addr_t phys, > >> + size_t size, enum dma_data_direction dir, > >> + unsigned long attrs) > >> +{ > >> + dma_addr_t dev_addr = xen_phys_to_dma(dev, phys); > > > > Yes, we need the xen_phys_to_dma call here. This is one of the reasons I > > don't think we can use dma_direct_map_resource() to implement > > map_resource > > > > > >> + BUG_ON(dir == DMA_NONE); > >> + > >> + if (!dma_capable(dev, dev_addr, size, false)) > >> + return DMA_MAPPING_ERROR; > > > > But here you also need to check that phys+size doesn't cross a page > > boundary. You need to call range_straddles_page_boundary, like we do at > > the beginning of xen_swiotlb_map_page. > > > > If it is possible to cross a page boundary, then we need to basically to > > do the same thing here as we do in xen_swiotlb_map_page where we check > > for: > > > > if (dma_capable(dev, dev_addr, size, true) && > > !range_straddles_page_boundary(phys, size) && > > !xen_arch_need_swiotlb(dev, phys, dev_addr) && > > !is_swiotlb_force_bounce(dev)) > > goto done; > > > > if all is well, we go with the native path, otherwise we bounce on > > swiotlb-xen. > > > > I'll preface this with that it's my first deep dive in DMA, so the > following may entirely be me being stupid: > > I'm not sure a straddles page boundary path makes sense. > > This mapping is not for a RAM backed address. In the eDMA case for the > iMX8QXP the `phys` coming in here is the address of a register. Ok, this information is important :-) I am not certain whether the map_resource interface can only be called for MMIO addresses or if it can also be called for RAM-backed addresses with a size > PAGE_SIZE. In the latter case, we could run into the issue I was describing. > I cannot see how a swiotlb bounce would fix anything if you end up > straddling a page boundary. At most I can see a WARN_ON with a > DMA_MAPPING_ERROR return if such a check would yield true. > > Let's say you want to do a DEV_TO_MEM and a check decides you need to > bounce it you'd bounce an RX register address. I cannot see how that DMA > would ever end up doing the expected thing. > > The eDMA engine will manage the RX/TX registers for an IP block and move > the data between them and RAM, where the RAM memory is mapped separately > by dma_map_page (and family). And these can be swiotlb bounced via the > map page callback with no problem. OK thanks for the explanation. Like I wrote above, if we are guaranteed that map_resource cannot be called for RAM-backed addresses or size is less than PAGE_SIZE, then your patch is good as-is. If there are no such guarantees, then we'll have to think a bit more about it. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] xen: swiotlb: Implement map_resource callback 2025-05-07 23:09 ` Stefano Stabellini @ 2025-05-08 4:14 ` Christoph Hellwig 2025-05-08 23:14 ` Stefano Stabellini 0 siblings, 1 reply; 11+ messages in thread From: Christoph Hellwig @ 2025-05-08 4:14 UTC (permalink / raw) To: Stefano Stabellini Cc: John Ernberg, Juergen Gross, Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev, Christoph Hellwig On Wed, May 07, 2025 at 04:09:15PM -0700, Stefano Stabellini wrote: > > This mapping is not for a RAM backed address. In the eDMA case for the > > iMX8QXP the `phys` coming in here is the address of a register. > > Ok, this information is important :-) > > I am not certain whether the map_resource interface can only be called > for MMIO addresses or if it can also be called for RAM-backed addresses > with a size > PAGE_SIZE. In the latter case, we could run into the issue > I was describing. map_resource is intended for MMIO regions, although those could be > PAGE_SIZE. It must not be called on RAM. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] xen: swiotlb: Implement map_resource callback 2025-05-08 4:14 ` Christoph Hellwig @ 2025-05-08 23:14 ` Stefano Stabellini 2025-05-09 7:48 ` John Ernberg 0 siblings, 1 reply; 11+ messages in thread From: Stefano Stabellini @ 2025-05-08 23:14 UTC (permalink / raw) To: Christoph Hellwig Cc: Stefano Stabellini, John Ernberg, Juergen Gross, Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev On Wed, 7 May 2025, Christoph Hellwig wrote: > On Wed, May 07, 2025 at 04:09:15PM -0700, Stefano Stabellini wrote: > > > This mapping is not for a RAM backed address. In the eDMA case for the > > > iMX8QXP the `phys` coming in here is the address of a register. > > > > Ok, this information is important :-) > > > > I am not certain whether the map_resource interface can only be called > > for MMIO addresses or if it can also be called for RAM-backed addresses > > with a size > PAGE_SIZE. In the latter case, we could run into the issue > > I was describing. > > map_resource is intended for MMIO regions, although those could be > > PAGE_SIZE. It must not be called on RAM. In that case, John, you can just use dma_direct_map_resource(). That's because MMIO regions: - are 1:1 mapped on ARM - are 1:1 mapped on x86 for PV Dom0 - might not be 1:1 mapped on x86 for PVH Dom0, but in this case we rely on the IOMMU to do address translation In none of these cases xen_phys_to_dma would give us any interesting results. It would be the same as calling phys_to_dma. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] xen: swiotlb: Implement map_resource callback 2025-05-08 23:14 ` Stefano Stabellini @ 2025-05-09 7:48 ` John Ernberg 0 siblings, 0 replies; 11+ messages in thread From: John Ernberg @ 2025-05-09 7:48 UTC (permalink / raw) To: Stefano Stabellini, Christoph Hellwig Cc: Juergen Gross, Oleksandr Tyshchenko, Catalin Marinas, Andrew Morton, xen-devel@lists.xenproject.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, imx@lists.linux.dev Hi Stefano, On 5/9/25 1:14 AM, Stefano Stabellini wrote: > On Wed, 7 May 2025, Christoph Hellwig wrote: >> On Wed, May 07, 2025 at 04:09:15PM -0700, Stefano Stabellini wrote: >>>> This mapping is not for a RAM backed address. In the eDMA case for the >>>> iMX8QXP the `phys` coming in here is the address of a register. >>> >>> Ok, this information is important :-) >>> >>> I am not certain whether the map_resource interface can only be called >>> for MMIO addresses or if it can also be called for RAM-backed addresses >>> with a size > PAGE_SIZE. In the latter case, we could run into the issue >>> I was describing. >> >> map_resource is intended for MMIO regions, although those could be > >> PAGE_SIZE. It must not be called on RAM. > > In that case, John, you can just use dma_direct_map_resource(). > > That's because MMIO regions: > - are 1:1 mapped on ARM > - are 1:1 mapped on x86 for PV Dom0 > - might not be 1:1 mapped on x86 for PVH Dom0, but in this case we rely > on the IOMMU to do address translation > > In none of these cases xen_phys_to_dma would give us any interesting > results. It would be the same as calling phys_to_dma. Thank you for explaining. I will spin a V2 incorporating this. Thanks! // John Ernberg ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2025-05-09 7:48 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-05-02 11:40 [PATCH 0/2] xen: swiotlb: 2 fixes SoCs w/o IOMMU (e.g. iMX8QXP) John Ernberg 2025-05-02 11:40 ` [PATCH 1/2] xen: swiotlb: Use swiotlb bouncing if kmalloc allocation demands it John Ernberg 2025-05-02 17:07 ` Stefano Stabellini 2025-05-08 15:21 ` Catalin Marinas 2025-05-02 11:40 ` [PATCH 2/2] xen: swiotlb: Implement map_resource callback John Ernberg 2025-05-02 17:20 ` Stefano Stabellini 2025-05-06 12:22 ` John Ernberg 2025-05-07 23:09 ` Stefano Stabellini 2025-05-08 4:14 ` Christoph Hellwig 2025-05-08 23:14 ` Stefano Stabellini 2025-05-09 7:48 ` John Ernberg
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox