[PATCH v2] dmaengine: cookie bypass for out of order completion

[PATCH v2] dmaengine: cookie bypass for out of order completion

[Dave Jiang]

The cookie tracking in dmaengine expects all submissions completed in
order. Some DMA devices like Intel DSA can complete submissions out of
order, especially if configured with a work queue sharing multiple DMA
engines. Add a status DMA_OUT_OF_ORDER that tx_status can be returned for
those DMA devices. The user should use callbacks to track the completion
rather than the DMA cookie. This would address the issue of dmatest
complaining that descriptors are "busy" when the cookie count goes
backwards due to out of order completion. Add DMA_NO_COMPLETION_ORDER
DMA capability to allow the driver to flag the device's ability to complete
operations out of order.

Reported-by: Swathi Kovvuri <swathi.kovvuri@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Swathi Kovvuri <swathi.kovvuri@intel.com>
---

v2:
- Add DMA capability (vinod)
- Add documentation (vinod)

 Documentation/driver-api/dmaengine/provider.rst |   11 +++++++++++
 drivers/dma/dmatest.c                           |    5 ++++-
 drivers/dma/idxd/dma.c                          |    3 ++-
 include/linux/dmaengine.h                       |    2 ++
 4 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/Documentation/driver-api/dmaengine/provider.rst b/Documentation/driver-api/dmaengine/provider.rst
index 56e5833e8a07..783ca141e147 100644
--- a/Documentation/driver-api/dmaengine/provider.rst
+++ b/Documentation/driver-api/dmaengine/provider.rst
@@ -239,6 +239,14 @@ Currently, the types available are:
     want to transfer a portion of uncompressed data directly to the
     display to print it
 
+- DMA_COMPLETION_NO_ORDER
+
+  - The device supports out of order completion of the operations.
+
+  - The driver should return DMA_OUT_OF_ORDER for device_tx_status if
+    the device supports out of order completion and the completion is
+    is expected to be completed out of order.
+
 These various types will also affect how the source and destination
 addresses change over time.
 
@@ -399,6 +407,9 @@ supported.
   - In the case of a cyclic transfer, it should only take into
     account the current period.
 
+  - Should return DMA_OUT_OF_ORDER if the device supports out of order
+    completion and is completing the operation out of order.
+
   - This function can be called in an interrupt context.
 
 - device_config
diff --git a/drivers/dma/dmatest.c b/drivers/dma/dmatest.c
index a2cadfa2e6d7..8953e096a05c 100644
--- a/drivers/dma/dmatest.c
+++ b/drivers/dma/dmatest.c
@@ -821,7 +821,10 @@ static int dmatest_func(void *data)
 			result("test timed out", total_tests, src->off, dst->off,
 			       len, 0);
 			goto error_unmap_continue;
-		} else if (status != DMA_COMPLETE) {
+		} else if (status != DMA_COMPLETE &&
+			   !(dma_has_cap(DMA_COMPLETION_NO_ORDER,
+					 dev->cap_mask) &&
+			     status == DMA_OUT_OF_ORDER)) {
 			result(status == DMA_ERROR ?
 			       "completion error status" :
 			       "completion busy status", total_tests, src->off,
diff --git a/drivers/dma/idxd/dma.c b/drivers/dma/idxd/dma.c
index c64c1429d160..8cce6de26116 100644
--- a/drivers/dma/idxd/dma.c
+++ b/drivers/dma/idxd/dma.c
@@ -133,7 +133,7 @@ static enum dma_status idxd_dma_tx_status(struct dma_chan *dma_chan,
 					  dma_cookie_t cookie,
 					  struct dma_tx_state *txstate)
 {
-	return dma_cookie_status(dma_chan, cookie, txstate);
+	return DMA_OUT_OF_ORDER;
 }
 
 /*
@@ -174,6 +174,7 @@ int idxd_register_dma_device(struct idxd_device *idxd)
 	INIT_LIST_HEAD(&dma->channels);
 	dma->dev = &idxd->pdev->dev;
 
+	dma_cap_set(DMA_NO_COMPLETION_ORDER, dma->cap_mask);
 	dma->device_release = idxd_dma_release;
 
 	if (idxd->hw.opcap.bits[0] & IDXD_OPCAP_MEMMOVE) {
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 21065c04c4ac..1123f4d15bae 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -39,6 +39,7 @@ enum dma_status {
 	DMA_IN_PROGRESS,
 	DMA_PAUSED,
 	DMA_ERROR,
+	DMA_OUT_OF_ORDER,
 };
 
 /**
@@ -61,6 +62,7 @@ enum dma_transaction_type {
 	DMA_SLAVE,
 	DMA_CYCLIC,
 	DMA_INTERLEAVE,
+	DMA_COMPLETION_NO_ORDER,
 /* last transaction type for creation of the capabilities mask */
 	DMA_TX_TYPE_END,
 };

Re: [PATCH v2] dmaengine: cookie bypass for out of order completion

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2986 bytes --]

Hi Dave,

I love your patch! Yet something to improve:

[auto build test ERROR on slave-dma/next]
[also build test ERROR on linus/master v5.7-rc4 next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Dave-Jiang/dmaengine-cookie-bypass-for-out-of-order-completion/20200506-115734
base:   https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/slave-dma.git next
config: x86_64-randconfig-g003-20200505 (attached as .config)
compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project 24b4965ce65b14ead595dcc68add22ba37533207)
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install x86_64 cross compiling tool for clang build
        # apt-get install binutils-x86-64-linux-gnu
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

>> drivers/dma/idxd/dma.c:177:14: error: use of undeclared identifier 'DMA_NO_COMPLETION_ORDER'; did you mean 'DMA_COMPLETION_NO_ORDER'?
           dma_cap_set(DMA_NO_COMPLETION_ORDER, dma->cap_mask);
                       ^~~~~~~~~~~~~~~~~~~~~~~
                       DMA_COMPLETION_NO_ORDER
   include/linux/dmaengine.h:1334:46: note: expanded from macro 'dma_cap_set'
   #define dma_cap_set(tx, mask) __dma_cap_set((tx), &(mask))
                                                ^
   include/linux/dmaengine.h:65:2: note: 'DMA_COMPLETION_NO_ORDER' declared here
           DMA_COMPLETION_NO_ORDER,
           ^
   1 error generated.

vim +177 drivers/dma/idxd/dma.c

   169	
   170	int idxd_register_dma_device(struct idxd_device *idxd)
   171	{
   172		struct dma_device *dma = &idxd->dma_dev;
   173	
   174		INIT_LIST_HEAD(&dma->channels);
   175		dma->dev = &idxd->pdev->dev;
   176	
 > 177		dma_cap_set(DMA_NO_COMPLETION_ORDER, dma->cap_mask);
   178		dma->device_release = idxd_dma_release;
   179	
   180		if (idxd->hw.opcap.bits[0] & IDXD_OPCAP_MEMMOVE) {
   181			dma_cap_set(DMA_MEMCPY, dma->cap_mask);
   182			dma->device_prep_dma_memcpy = idxd_dma_submit_memcpy;
   183		}
   184	
   185		dma->device_tx_status = idxd_dma_tx_status;
   186		dma->device_issue_pending = idxd_dma_issue_pending;
   187		dma->device_alloc_chan_resources = idxd_dma_alloc_chan_resources;
   188		dma->device_free_chan_resources = idxd_dma_free_chan_resources;
   189	
   190		return dma_async_device_register(&idxd->dma_dev);
   191	}
   192	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 35776 bytes --]

Re: [PATCH v2] dmaengine: cookie bypass for out of order completion

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2911 bytes --]

Hi Dave,

I love your patch! Yet something to improve:

[auto build test ERROR on slave-dma/next]
[also build test ERROR on linus/master v5.7-rc4 next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Dave-Jiang/dmaengine-cookie-bypass-for-out-of-order-completion/20200506-115734
base:   https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/slave-dma.git next
config: x86_64-randconfig-a003-20200506 (attached as .config)
compiler: gcc-7 (Ubuntu 7.5.0-6ubuntu2) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from drivers/dma/idxd/dma.c:9:0:
   drivers/dma/idxd/dma.c: In function 'idxd_register_dma_device':
>> drivers/dma/idxd/dma.c:177:14: error: 'DMA_NO_COMPLETION_ORDER' undeclared (first use in this function); did you mean 'DMA_COMPLETION_NO_ORDER'?
     dma_cap_set(DMA_NO_COMPLETION_ORDER, dma->cap_mask);
                 ^
   include/linux/dmaengine.h:1334:46: note: in definition of macro 'dma_cap_set'
    #define dma_cap_set(tx, mask) __dma_cap_set((tx), &(mask))
                                                 ^~
   drivers/dma/idxd/dma.c:177:14: note: each undeclared identifier is reported only once for each function it appears in
     dma_cap_set(DMA_NO_COMPLETION_ORDER, dma->cap_mask);
                 ^
   include/linux/dmaengine.h:1334:46: note: in definition of macro 'dma_cap_set'
    #define dma_cap_set(tx, mask) __dma_cap_set((tx), &(mask))
                                                 ^~

vim +177 drivers/dma/idxd/dma.c

   169	
   170	int idxd_register_dma_device(struct idxd_device *idxd)
   171	{
   172		struct dma_device *dma = &idxd->dma_dev;
   173	
   174		INIT_LIST_HEAD(&dma->channels);
   175		dma->dev = &idxd->pdev->dev;
   176	
 > 177		dma_cap_set(DMA_NO_COMPLETION_ORDER, dma->cap_mask);
   178		dma->device_release = idxd_dma_release;
   179	
   180		if (idxd->hw.opcap.bits[0] & IDXD_OPCAP_MEMMOVE) {
   181			dma_cap_set(DMA_MEMCPY, dma->cap_mask);
   182			dma->device_prep_dma_memcpy = idxd_dma_submit_memcpy;
   183		}
   184	
   185		dma->device_tx_status = idxd_dma_tx_status;
   186		dma->device_issue_pending = idxd_dma_issue_pending;
   187		dma->device_alloc_chan_resources = idxd_dma_alloc_chan_resources;
   188		dma->device_free_chan_resources = idxd_dma_free_chan_resources;
   189	
   190		return dma_async_device_register(&idxd->dma_dev);
   191	}
   192	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 33720 bytes --]