[PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge boundary

[PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge boundary

[Christoph Hellwig]

Only call into nvme_alloc_host_mem_single which uses
dma_alloc_noncontiguous when there is non-null dma merge boundary.
Without this we'll call into dma_alloc_noncontiguous for device using
dma-direct, which can work fine as long as the preferred size is below the
MAX_ORDER of the page allocator, but blows up with a warning if it is
too large.

Fixes: 63a5c7a4b4c4 ("nvme-pci: use dma_alloc_noncontigous if possible")
Reported-by: Leon Romanovsky <leon@kernel.org>
Reported-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/nvme/host/pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 4c644bb7f069..778f124c2e21 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2172,6 +2172,7 @@ static int nvme_alloc_host_mem_multi(struct nvme_dev *dev, u64 preferred,
 
 static int nvme_alloc_host_mem(struct nvme_dev *dev, u64 min, u64 preferred)
 {
+	unsigned long dma_merge_moundary = dma_get_merge_boundary(dev->dev);
 	u64 min_chunk = min_t(u64, preferred, PAGE_SIZE * MAX_ORDER_NR_PAGES);
 	u64 hmminds = max_t(u32, dev->ctrl.hmminds * 4096, PAGE_SIZE * 2);
 	u64 chunk_size;
@@ -2180,7 +2181,7 @@ static int nvme_alloc_host_mem(struct nvme_dev *dev, u64 min, u64 preferred)
 	 * If there is an IOMMU that can merge pages, try a virtually
 	 * non-contiguous allocation for a single segment first.
 	 */
-	if (!(PAGE_SIZE & dma_get_merge_boundary(dev->dev))) {
+	if (dma_merge_moundary && (PAGE_SIZE & dma_merge_moundary) == 0) {
 		if (!nvme_alloc_host_mem_single(dev, preferred))
 			return 0;
 	}
-- 
2.45.2

Re: [PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge boundary

[Jens Axboe]

On 12/3/24 5:42 PM, Christoph Hellwig wrote:
> Only call into nvme_alloc_host_mem_single which uses
> dma_alloc_noncontiguous when there is non-null dma merge boundary.
> Without this we'll call into dma_alloc_noncontiguous for device using
> dma-direct, which can work fine as long as the preferred size is below the
> MAX_ORDER of the page allocator, but blows up with a warning if it is
> too large.
> 
> Fixes: 63a5c7a4b4c4 ("nvme-pci: use dma_alloc_noncontigous if possible")
> Reported-by: Leon Romanovsky <leon@kernel.org>
> Reported-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  drivers/nvme/host/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 4c644bb7f069..778f124c2e21 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -2172,6 +2172,7 @@ static int nvme_alloc_host_mem_multi(struct nvme_dev *dev, u64 preferred,
>  
>  static int nvme_alloc_host_mem(struct nvme_dev *dev, u64 min, u64 preferred)
>  {
> +	unsigned long dma_merge_moundary = dma_get_merge_boundary(dev->dev);

Fix looks fine, but s/moundary/boundary?

-- 
Jens Axboe

RE: [PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge boundary

[Borah, Chaitanya Kumar]

> -----Original Message-----
> From: Christoph Hellwig <hch@lst.de>
> Sent: Wednesday, December 4, 2024 6:13 AM
> To: kbusch@kernel.org; sagi@grimberg.me
> Cc: linux-nvme@lists.infradead.org; Leon Romanovsky <leon@kernel.org>;
> Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>
> Subject: [PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge
> boundary
> 
> Only call into nvme_alloc_host_mem_single which uses
> dma_alloc_noncontiguous when there is non-null dma merge boundary.
> Without this we'll call into dma_alloc_noncontiguous for device using dma-
> direct, which can work fine as long as the preferred size is below the
> MAX_ORDER of the page allocator, but blows up with a warning if it is too
> large.
> 
> Fixes: 63a5c7a4b4c4 ("nvme-pci: use dma_alloc_noncontigous if possible")
> Reported-by: Leon Romanovsky <leon@kernel.org>
> Reported-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>


Tested-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>

> ---
>  drivers/nvme/host/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index
> 4c644bb7f069..778f124c2e21 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -2172,6 +2172,7 @@ static int nvme_alloc_host_mem_multi(struct
> nvme_dev *dev, u64 preferred,
> 
>  static int nvme_alloc_host_mem(struct nvme_dev *dev, u64 min, u64
> preferred)  {
> +	unsigned long dma_merge_moundary =
> dma_get_merge_boundary(dev->dev);
>  	u64 min_chunk = min_t(u64, preferred, PAGE_SIZE *
> MAX_ORDER_NR_PAGES);
>  	u64 hmminds = max_t(u32, dev->ctrl.hmminds * 4096, PAGE_SIZE *
> 2);
>  	u64 chunk_size;
> @@ -2180,7 +2181,7 @@ static int nvme_alloc_host_mem(struct nvme_dev
> *dev, u64 min, u64 preferred)
>  	 * If there is an IOMMU that can merge pages, try a virtually
>  	 * non-contiguous allocation for a single segment first.
>  	 */
> -	if (!(PAGE_SIZE & dma_get_merge_boundary(dev->dev))) {
> +	if (dma_merge_moundary && (PAGE_SIZE & dma_merge_moundary)
> == 0) {
>  		if (!nvme_alloc_host_mem_single(dev, preferred))
>  			return 0;
>  	}
> --
> 2.45.2

Re: [PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge boundary

[Leon Romanovsky]

On Wed, Dec 04, 2024 at 09:42:40AM +0900, Christoph Hellwig wrote:
> Only call into nvme_alloc_host_mem_single which uses
> dma_alloc_noncontiguous when there is non-null dma merge boundary.
> Without this we'll call into dma_alloc_noncontiguous for device using
> dma-direct, which can work fine as long as the preferred size is below the
> MAX_ORDER of the page allocator, but blows up with a warning if it is
> too large.
> 
> Fixes: 63a5c7a4b4c4 ("nvme-pci: use dma_alloc_noncontigous if possible")
> Reported-by: Leon Romanovsky <leon@kernel.org>
> Reported-by: Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  drivers/nvme/host/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 

Except moundary vs. boundary, everything looks good.

Thanks for the fix.
Reviewed-by: Leon Romanovsky <leon@kernel.org>

Re: [PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge boundary

[Keith Busch]

On Wed, Dec 04, 2024 at 09:42:40AM +0900, Christoph Hellwig wrote:
> Only call into nvme_alloc_host_mem_single which uses
> dma_alloc_noncontiguous when there is non-null dma merge boundary.
> Without this we'll call into dma_alloc_noncontiguous for device using
> dma-direct, which can work fine as long as the preferred size is below the
> MAX_ORDER of the page allocator, but blows up with a warning if it is
> too large.

Thanks, applied to nvme-6.13 with the mentioned spelling fix.

Re: [PATCH] nvme-pci: don't use dma_alloc_noncontiguous with 0 merge boundary

[Christoph Hellwig]

On Tue, Dec 03, 2024 at 06:36:08PM -0700, Jens Axboe wrote:
> Fix looks fine, but s/moundary/boundary?

Yes :)