[PATCH 0/3] dma, nvme, powerpc: introduce and implement DMA_ATTR_NO_WARN

[PATCH 0/3] dma, nvme, powerpc: introduce and implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

This patchset introduces dma_attr DMA_ATTR_NO_WARN (just like __GFP_NOWARN),
which tells the DMA-mapping subsystem to supress allocation failure reports.

On some architectures allocation failures are reported with error messages
to the system logs.  Although this can help to identify and debug problems,
drivers which handle failures (eg, retry later) have no problems with them,
and can actually flood the system logs with error messages that aren't any
problem at all, depending on the implementation of the retry mechanism.

So, this provides a way for drivers to avoid those error messages on calls
where allocation failures are not a problem, and shouldn't bother the logs.

 - Patch 1/3 introduces and documents the new dma_attr.

 - Patch 2/3 implements it on the nvme driver (which might repeatedly trip
             on allocation failures due to high load, flooding system logs
             with error messages at least on powerpc: "iommu_alloc failed")

 - Patch 3/3 implements support for it on powerpc arch (where this problem
             was observed.  It's possible to extend support for more archs
             if the patchset is welcome).

Mauricio Faria de Oliveira (3):
  dma: introduce DMA_ATTR_NO_WARN
  nvme: implement DMA_ATTR_NO_WARN
  powerpc: implement DMA_ATTR_NO_WARN

 Documentation/DMA-attributes.txt | 17 +++++++++++++++++
 arch/powerpc/kernel/iommu.c      |  4 ++--
 drivers/nvme/host/pci.c          | 10 ++++++++--
 include/linux/dma-attrs.h        |  1 +
 4 files changed, 28 insertions(+), 4 deletions(-)

-- 
1.8.3.1

[PATCH 0/3] dma, nvme, powerpc: introduce and implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

This patchset introduces dma_attr DMA_ATTR_NO_WARN (just like __GFP_NOWARN),
which tells the DMA-mapping subsystem to supress allocation failure reports.

On some architectures allocation failures are reported with error messages
to the system logs.  Although this can help to identify and debug problems,
drivers which handle failures (eg, retry later) have no problems with them,
and can actually flood the system logs with error messages that aren't any
problem at all, depending on the implementation of the retry mechanism.

So, this provides a way for drivers to avoid those error messages on calls
where allocation failures are not a problem, and shouldn't bother the logs.

 - Patch 1/3 introduces and documents the new dma_attr.

 - Patch 2/3 implements it on the nvme driver (which might repeatedly trip
             on allocation failures due to high load, flooding system logs
             with error messages at least on powerpc: "iommu_alloc failed")

 - Patch 3/3 implements support for it on powerpc arch (where this problem
             was observed.  It's possible to extend support for more archs
             if the patchset is welcome).

Mauricio Faria de Oliveira (3):
  dma: introduce DMA_ATTR_NO_WARN
  nvme: implement DMA_ATTR_NO_WARN
  powerpc: implement DMA_ATTR_NO_WARN

 Documentation/DMA-attributes.txt | 17 +++++++++++++++++
 arch/powerpc/kernel/iommu.c      |  4 ++--
 drivers/nvme/host/pci.c          | 10 ++++++++--
 include/linux/dma-attrs.h        |  1 +
 4 files changed, 28 insertions(+), 4 deletions(-)

-- 
1.8.3.1

[PATCH 1/3] dma: introduce DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

Introduce the DMA_ATTR_NO_WARN attribute, and document it.

Signed-off-by: Mauricio Faria de Oliveira <mauricfo at linux.vnet.ibm.com>
---
 Documentation/DMA-attributes.txt | 17 +++++++++++++++++
 include/linux/dma-attrs.h        |  1 +
 2 files changed, 18 insertions(+)

diff --git a/Documentation/DMA-attributes.txt b/Documentation/DMA-attributes.txt
index e8cf9cf..841e771 100644
--- a/Documentation/DMA-attributes.txt
+++ b/Documentation/DMA-attributes.txt
@@ -126,3 +126,20 @@ means that we won't try quite as hard to get them.
 
 NOTE: At the moment DMA_ATTR_ALLOC_SINGLE_PAGES is only implemented on ARM,
 though ARM64 patches will likely be posted soon.
+
+DMA_ATTR_NO_WARN
+----------------
+
+This tells the DMA-mapping subsystem to supress allocation failure reports
+(similarly to __GFP_NOWARN).
+
+On some architectures allocation failures are reported with error messages
+to the system logs.  Although this can help to identify and debug problems,
+drivers which handle failures (eg, retry later) have no problems with them,
+and can actually flood the system logs with error messages that aren't any
+problem at all, depending on the implementation of the retry mechanism.
+
+So, this provides a way for drivers to avoid those error messages on calls
+where allocation failures are not a problem, and shouldn't bother the logs.
+
+NOTE: At the moment DMA_ATTR_NO_WARN is only implemented on PowerPC.
diff --git a/include/linux/dma-attrs.h b/include/linux/dma-attrs.h
index f3c5aea..0577389 100644
--- a/include/linux/dma-attrs.h
+++ b/include/linux/dma-attrs.h
@@ -19,6 +19,7 @@ enum dma_attr {
 	DMA_ATTR_SKIP_CPU_SYNC,
 	DMA_ATTR_FORCE_CONTIGUOUS,
 	DMA_ATTR_ALLOC_SINGLE_PAGES,
+	DMA_ATTR_NO_WARN,
 	DMA_ATTR_MAX,
 };
 
-- 
1.8.3.1

[PATCH 1/3] dma: introduce DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

Introduce the DMA_ATTR_NO_WARN attribute, and document it.

Signed-off-by: Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com>
---
 Documentation/DMA-attributes.txt | 17 +++++++++++++++++
 include/linux/dma-attrs.h        |  1 +
 2 files changed, 18 insertions(+)

diff --git a/Documentation/DMA-attributes.txt b/Documentation/DMA-attributes.txt
index e8cf9cf..841e771 100644
--- a/Documentation/DMA-attributes.txt
+++ b/Documentation/DMA-attributes.txt
@@ -126,3 +126,20 @@ means that we won't try quite as hard to get them.
 
 NOTE: At the moment DMA_ATTR_ALLOC_SINGLE_PAGES is only implemented on ARM,
 though ARM64 patches will likely be posted soon.
+
+DMA_ATTR_NO_WARN
+----------------
+
+This tells the DMA-mapping subsystem to supress allocation failure reports
+(similarly to __GFP_NOWARN).
+
+On some architectures allocation failures are reported with error messages
+to the system logs.  Although this can help to identify and debug problems,
+drivers which handle failures (eg, retry later) have no problems with them,
+and can actually flood the system logs with error messages that aren't any
+problem at all, depending on the implementation of the retry mechanism.
+
+So, this provides a way for drivers to avoid those error messages on calls
+where allocation failures are not a problem, and shouldn't bother the logs.
+
+NOTE: At the moment DMA_ATTR_NO_WARN is only implemented on PowerPC.
diff --git a/include/linux/dma-attrs.h b/include/linux/dma-attrs.h
index f3c5aea..0577389 100644
--- a/include/linux/dma-attrs.h
+++ b/include/linux/dma-attrs.h
@@ -19,6 +19,7 @@ enum dma_attr {
 	DMA_ATTR_SKIP_CPU_SYNC,
 	DMA_ATTR_FORCE_CONTIGUOUS,
 	DMA_ATTR_ALLOC_SINGLE_PAGES,
+	DMA_ATTR_NO_WARN,
 	DMA_ATTR_MAX,
 };
 
-- 
1.8.3.1

[PATCH 2/3] nvme: implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

Use the DMA_ATTR_NO_WARN attribute on dma_map_sg() calls of nvme driver.

Signed-off-by: Mauricio Faria de Oliveira <mauricfo at linux.vnet.ibm.com>
---
 drivers/nvme/host/pci.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index d1a8259..c3c3348 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -18,6 +18,7 @@
 #include <linux/blk-mq.h>
 #include <linux/cpu.h>
 #include <linux/delay.h>
+#include <linux/dma-attrs.h>
 #include <linux/errno.h>
 #include <linux/fs.h>
 #include <linux/genhd.h>
@@ -65,6 +66,8 @@ MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
 
 static struct workqueue_struct *nvme_workq;
 
+static DEFINE_DMA_ATTRS(nvme_dma_attrs);
+
 struct nvme_dev;
 struct nvme_queue;
 
@@ -498,7 +501,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
 		goto out;
 
 	ret = BLK_MQ_RQ_QUEUE_BUSY;
-	if (!dma_map_sg(dev->dev, iod->sg, iod->nents, dma_dir))
+	if (!dma_map_sg_attrs(dev->dev, iod->sg, iod->nents, dma_dir, &nvme_dma_attrs))
 		goto out;
 
 	if (!nvme_setup_prps(dev, req, size))
@@ -516,7 +519,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
 		if (rq_data_dir(req))
 			nvme_dif_remap(req, nvme_dif_prep);
 
-		if (!dma_map_sg(dev->dev, &iod->meta_sg, 1, dma_dir))
+		if (!dma_map_sg_attrs(dev->dev, &iod->meta_sg, 1, dma_dir, &nvme_dma_attrs))
 			goto out_unmap;
 	}
 
@@ -2118,6 +2121,9 @@ static int __init nvme_init(void)
 	result = pci_register_driver(&nvme_driver);
 	if (result)
 		destroy_workqueue(nvme_workq);
+
+	dma_set_attr(DMA_ATTR_NO_WARN, &nvme_dma_attrs);
+
 	return result;
 }
 
-- 
1.8.3.1

[PATCH 2/3] nvme: implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

Use the DMA_ATTR_NO_WARN attribute on dma_map_sg() calls of nvme driver.

Signed-off-by: Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com>
---
 drivers/nvme/host/pci.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index d1a8259..c3c3348 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -18,6 +18,7 @@
 #include <linux/blk-mq.h>
 #include <linux/cpu.h>
 #include <linux/delay.h>
+#include <linux/dma-attrs.h>
 #include <linux/errno.h>
 #include <linux/fs.h>
 #include <linux/genhd.h>
@@ -65,6 +66,8 @@ MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
 
 static struct workqueue_struct *nvme_workq;
 
+static DEFINE_DMA_ATTRS(nvme_dma_attrs);
+
 struct nvme_dev;
 struct nvme_queue;
 
@@ -498,7 +501,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
 		goto out;
 
 	ret = BLK_MQ_RQ_QUEUE_BUSY;
-	if (!dma_map_sg(dev->dev, iod->sg, iod->nents, dma_dir))
+	if (!dma_map_sg_attrs(dev->dev, iod->sg, iod->nents, dma_dir, &nvme_dma_attrs))
 		goto out;
 
 	if (!nvme_setup_prps(dev, req, size))
@@ -516,7 +519,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
 		if (rq_data_dir(req))
 			nvme_dif_remap(req, nvme_dif_prep);
 
-		if (!dma_map_sg(dev->dev, &iod->meta_sg, 1, dma_dir))
+		if (!dma_map_sg_attrs(dev->dev, &iod->meta_sg, 1, dma_dir, &nvme_dma_attrs))
 			goto out_unmap;
 	}
 
@@ -2118,6 +2121,9 @@ static int __init nvme_init(void)
 	result = pci_register_driver(&nvme_driver);
 	if (result)
 		destroy_workqueue(nvme_workq);
+
+	dma_set_attr(DMA_ATTR_NO_WARN, &nvme_dma_attrs);
+
 	return result;
 }
 
-- 
1.8.3.1

[PATCH 2/3] nvme: implement DMA_ATTR_NO_WARN

[Gabriel Krisman Bertazi]

Mauricio Faria de Oliveira <mauricfo at linux.vnet.ibm.com> writes:

> Use the DMA_ATTR_NO_WARN attribute on dma_map_sg() calls of nvme driver.
>
> Signed-off-by: Mauricio Faria de Oliveira <mauricfo at linux.vnet.ibm.com>

checkpatch.pl complains about line wrapping.  Other than that, this
looks good to me.

Reviewed-by: Gabriel Krisman Bertazi <krisman at linux.vnet.ibm.com>

> ---
>  drivers/nvme/host/pci.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index d1a8259..c3c3348 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -18,6 +18,7 @@
>  #include <linux/blk-mq.h>
>  #include <linux/cpu.h>
>  #include <linux/delay.h>
> +#include <linux/dma-attrs.h>
>  #include <linux/errno.h>
>  #include <linux/fs.h>
>  #include <linux/genhd.h>
> @@ -65,6 +66,8 @@ MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
>
>  static struct workqueue_struct *nvme_workq;
>
> +static DEFINE_DMA_ATTRS(nvme_dma_attrs);
> +
>  struct nvme_dev;
>  struct nvme_queue;
>
> @@ -498,7 +501,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
>  		goto out;
>
>  	ret = BLK_MQ_RQ_QUEUE_BUSY;
> -	if (!dma_map_sg(dev->dev, iod->sg, iod->nents, dma_dir))
> +	if (!dma_map_sg_attrs(dev->dev, iod->sg, iod->nents, dma_dir, &nvme_dma_attrs))
>  		goto out;
>
>  	if (!nvme_setup_prps(dev, req, size))
> @@ -516,7 +519,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
>  		if (rq_data_dir(req))
>  			nvme_dif_remap(req, nvme_dif_prep);
>
> -		if (!dma_map_sg(dev->dev, &iod->meta_sg, 1, dma_dir))
> +		if (!dma_map_sg_attrs(dev->dev, &iod->meta_sg, 1, dma_dir, &nvme_dma_attrs))
>  			goto out_unmap;
>  	}
>
> @@ -2118,6 +2121,9 @@ static int __init nvme_init(void)
>  	result = pci_register_driver(&nvme_driver);
>  	if (result)
>  		destroy_workqueue(nvme_workq);
> +
> +	dma_set_attr(DMA_ATTR_NO_WARN, &nvme_dma_attrs);
> +
>  	return result;
>  }

-- 
Gabriel Krisman Bertazi

Re: [PATCH 2/3] nvme: implement DMA_ATTR_NO_WARN

[Gabriel Krisman Bertazi]

Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com> writes:

> Use the DMA_ATTR_NO_WARN attribute on dma_map_sg() calls of nvme driver.
>
> Signed-off-by: Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com>

checkpatch.pl complains about line wrapping.  Other than that, this
looks good to me.

Reviewed-by: Gabriel Krisman Bertazi <krisman@linux.vnet.ibm.com>

> ---
>  drivers/nvme/host/pci.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index d1a8259..c3c3348 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -18,6 +18,7 @@
>  #include <linux/blk-mq.h>
>  #include <linux/cpu.h>
>  #include <linux/delay.h>
> +#include <linux/dma-attrs.h>
>  #include <linux/errno.h>
>  #include <linux/fs.h>
>  #include <linux/genhd.h>
> @@ -65,6 +66,8 @@ MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
>
>  static struct workqueue_struct *nvme_workq;
>
> +static DEFINE_DMA_ATTRS(nvme_dma_attrs);
> +
>  struct nvme_dev;
>  struct nvme_queue;
>
> @@ -498,7 +501,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
>  		goto out;
>
>  	ret = BLK_MQ_RQ_QUEUE_BUSY;
> -	if (!dma_map_sg(dev->dev, iod->sg, iod->nents, dma_dir))
> +	if (!dma_map_sg_attrs(dev->dev, iod->sg, iod->nents, dma_dir, &nvme_dma_attrs))
>  		goto out;
>
>  	if (!nvme_setup_prps(dev, req, size))
> @@ -516,7 +519,7 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
>  		if (rq_data_dir(req))
>  			nvme_dif_remap(req, nvme_dif_prep);
>
> -		if (!dma_map_sg(dev->dev, &iod->meta_sg, 1, dma_dir))
> +		if (!dma_map_sg_attrs(dev->dev, &iod->meta_sg, 1, dma_dir, &nvme_dma_attrs))
>  			goto out_unmap;
>  	}
>
> @@ -2118,6 +2121,9 @@ static int __init nvme_init(void)
>  	result = pci_register_driver(&nvme_driver);
>  	if (result)
>  		destroy_workqueue(nvme_workq);
> +
> +	dma_set_attr(DMA_ATTR_NO_WARN, &nvme_dma_attrs);
> +
>  	return result;
>  }

-- 
Gabriel Krisman Bertazi

[PATCH 2/3] nvme: implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

On 07/06/2016 09:41 PM, Gabriel Krisman Bertazi wrote:
> checkpatch.pl complains about line wrapping.  Other than that, this
> looks good to me.

I'll submit a v2 w/ that and typos fixed.
Thanks for reviewing.

-- 
Mauricio Faria de Oliveira
IBM Linux Technology Center

Re: [PATCH 2/3] nvme: implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

On 07/06/2016 09:41 PM, Gabriel Krisman Bertazi wrote:
> checkpatch.pl complains about line wrapping.  Other than that, this
> looks good to me.

I'll submit a v2 w/ that and typos fixed.
Thanks for reviewing.

-- 
Mauricio Faria de Oliveira
IBM Linux Technology Center

[PATCH 3/3] powerpc: implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

Add support for the DMA_ATTR_NO_WARN attribute on powerpc iommu code.

Signed-off-by: Mauricio Faria de Oliveira <mauricfo at linux.vnet.ibm.com>
---
 arch/powerpc/kernel/iommu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index a8e3490..69bb17f 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -479,7 +479,7 @@ int ppc_iommu_map_sg(struct device *dev, struct iommu_table *tbl,
 
 		/* Handle failure */
 		if (unlikely(entry == DMA_ERROR_CODE)) {
-			if (printk_ratelimit())
+			if (unlikely(!dma_get_attr(DMA_ATTR_NO_WARN, attrs)) && printk_ratelimit())
 				dev_info(dev, "iommu_alloc failed, tbl %p "
 					 "vaddr %lx npages %lu\n", tbl, vaddr,
 					 npages);
@@ -776,7 +776,7 @@ dma_addr_t iommu_map_page(struct device *dev, struct iommu_table *tbl,
 					 mask >> tbl->it_page_shift, align,
 					 attrs);
 		if (dma_handle == DMA_ERROR_CODE) {
-			if (printk_ratelimit())  {
+			if (unlikely(!dma_get_attr(DMA_ATTR_NO_WARN, attrs)) && printk_ratelimit())  {
 				dev_info(dev, "iommu_alloc failed, tbl %p "
 					 "vaddr %p npages %d\n", tbl, vaddr,
 					 npages);
-- 
1.8.3.1

[PATCH 3/3] powerpc: implement DMA_ATTR_NO_WARN

[Mauricio Faria de Oliveira]

Add support for the DMA_ATTR_NO_WARN attribute on powerpc iommu code.

Signed-off-by: Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com>
---
 arch/powerpc/kernel/iommu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index a8e3490..69bb17f 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -479,7 +479,7 @@ int ppc_iommu_map_sg(struct device *dev, struct iommu_table *tbl,
 
 		/* Handle failure */
 		if (unlikely(entry == DMA_ERROR_CODE)) {
-			if (printk_ratelimit())
+			if (unlikely(!dma_get_attr(DMA_ATTR_NO_WARN, attrs)) && printk_ratelimit())
 				dev_info(dev, "iommu_alloc failed, tbl %p "
 					 "vaddr %lx npages %lu\n", tbl, vaddr,
 					 npages);
@@ -776,7 +776,7 @@ dma_addr_t iommu_map_page(struct device *dev, struct iommu_table *tbl,
 					 mask >> tbl->it_page_shift, align,
 					 attrs);
 		if (dma_handle == DMA_ERROR_CODE) {
-			if (printk_ratelimit())  {
+			if (unlikely(!dma_get_attr(DMA_ATTR_NO_WARN, attrs)) && printk_ratelimit())  {
 				dev_info(dev, "iommu_alloc failed, tbl %p "
 					 "vaddr %p npages %d\n", tbl, vaddr,
 					 npages);
-- 
1.8.3.1