[PATCH v4 0/2] Some fixes for idxd driver

[PATCH v4 0/2] Some fixes for idxd driver

['Guanjun']

From: Guanjun <guanjun@linux.alibaba.com>

Hi Dave, Fenghua,
As we talked in v1 and v2, I add fixes tag in patch 0 and change some
descriptions in patch 1.
If there are no other issues, please queue it up.

Thanks,
Guanjun

Guanjun (2):
  dmaengine: idxd: Protect int_handle field in hw descriptor
  dmaengine: idxd: Fix incorrect descriptions for GRPCFG register

 drivers/dma/idxd/registers.h | 12 +++++++-----
 drivers/dma/idxd/submit.c    | 14 +++++++-------
 2 files changed, 14 insertions(+), 12 deletions(-)

-- 
2.39.3

[PATCH v4 1/2] dmaengine: idxd: Protect int_handle field in hw descriptor

['Guanjun']

From: Guanjun <guanjun@linux.alibaba.com>

The int_handle field in hw descriptor should also be protected
by wmb() before possibly triggering a DMA read.

Fixes: ec0d64231615 (dmaengine: idxd: embed irq_entry in idxd_wq struct)
Signed-off-by: Guanjun <guanjun@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghua.yu@intel.com>
---
 drivers/dma/idxd/submit.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/idxd/submit.c b/drivers/dma/idxd/submit.c
index c01db23e3333..3f922518e3a5 100644
--- a/drivers/dma/idxd/submit.c
+++ b/drivers/dma/idxd/submit.c
@@ -182,13 +182,6 @@ int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc)
 
 	portal = idxd_wq_portal_addr(wq);
 
-	/*
-	 * The wmb() flushes writes to coherent DMA data before
-	 * possibly triggering a DMA read. The wmb() is necessary
-	 * even on UP because the recipient is a device.
-	 */
-	wmb();
-
 	/*
 	 * Pending the descriptor to the lockless list for the irq_entry
 	 * that we designated the descriptor to.
@@ -199,6 +192,13 @@ int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc)
 		llist_add(&desc->llnode, &ie->pending_llist);
 	}
 
+	/*
+	 * The wmb() flushes writes to coherent DMA data before
+	 * possibly triggering a DMA read. The wmb() is necessary
+	 * even on UP because the recipient is a device.
+	 */
+	wmb();
+
 	if (wq_dedicated(wq)) {
 		iosubmit_cmds512(portal, desc->hw, 1);
 	} else {
-- 
2.39.3

[PATCH v4 2/2] dmaengine: idxd: Fix incorrect descriptions for GRPCFG register

['Guanjun']

From: Guanjun <guanjun@linux.alibaba.com>

Fix incorrect descriptions for the GRPCFG register which has three
sub-registers (GRPWQCFG, GRPENGCFG and GRPFLGCFG).
No functional changes

Signed-off-by: Guanjun <guanjun@linux.alibaba.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghua.yu@intel.com>
---
 drivers/dma/idxd/registers.h | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h
index 7b54a3939ea1..315c004f58e4 100644
--- a/drivers/dma/idxd/registers.h
+++ b/drivers/dma/idxd/registers.h
@@ -440,12 +440,14 @@ union wqcfg {
 /*
  * This macro calculates the offset into the GRPCFG register
  * idxd - struct idxd *
- * n - wq id
- * ofs - the index of the 32b dword for the config register
+ * n - group id
+ * ofs - the index of the 64b qword for the config register
  *
- * The WQCFG register block is divided into groups per each wq. The n index
- * allows us to move to the register group that's for that particular wq.
- * Each register is 32bits. The ofs gives us the number of register to access.
+ * The GRPCFG register block is divided into three sub-registers, which
+ * are GRPWQCFG, GRPENGCFG and GRPFLGCFG. The n index allows us to move
+ * to the register block that contains the three sub-registers.
+ * Each register block is 64bits. And the ofs gives us the offset
+ * within the GRPWQCFG register to access.
  */
 #define GRPWQCFG_OFFSET(idxd_dev, n, ofs) ((idxd_dev)->grpcfg_offset +\
 					   (n) * GRPCFG_SIZE + sizeof(u64) * (ofs))
-- 
2.39.3

Re: [PATCH v4 1/2] dmaengine: idxd: Protect int_handle field in hw descriptor

[Lijun Pan]

On 10/30/2023 9:55 PM, 'Guanjun' wrote:
> From: Guanjun <guanjun@linux.alibaba.com>
> 
> The int_handle field in hw descriptor should also be protected
> by wmb() before possibly triggering a DMA read.
> 
> Fixes: ec0d64231615 (dmaengine: idxd: embed irq_entry in idxd_wq struct)

I think the direct fix is to eb0cf33a91b4 which moves the interrupt 
handle assignment to idxd_submit_desc() and has a write to 
desc->hw->int_handle before submission of desc->hw.

Fixes: eb0cf33a91b4 ("dmaengine: idxd: move interrupt handle assignment")

Other than that,
Reviewed-by: Lijun Pan <lijun.pan@intel.com>


> Signed-off-by: Guanjun <guanjun@linux.alibaba.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com>
> ---
>   drivers/dma/idxd/submit.c | 14 +++++++-------
>   1 file changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/dma/idxd/submit.c b/drivers/dma/idxd/submit.c
> index c01db23e3333..3f922518e3a5 100644
> --- a/drivers/dma/idxd/submit.c
> +++ b/drivers/dma/idxd/submit.c
> @@ -182,13 +182,6 @@ int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc)
>   
>   	portal = idxd_wq_portal_addr(wq);
>   
> -	/*
> -	 * The wmb() flushes writes to coherent DMA data before
> -	 * possibly triggering a DMA read. The wmb() is necessary
> -	 * even on UP because the recipient is a device.
> -	 */
> -	wmb();
> -
>   	/*
>   	 * Pending the descriptor to the lockless list for the irq_entry
>   	 * that we designated the descriptor to.
> @@ -199,6 +192,13 @@ int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc)
>   		llist_add(&desc->llnode, &ie->pending_llist);
>   	}
>   
> +	/*
> +	 * The wmb() flushes writes to coherent DMA data before
> +	 * possibly triggering a DMA read. The wmb() is necessary
> +	 * even on UP because the recipient is a device.
> +	 */
> +	wmb();
> +
>   	if (wq_dedicated(wq)) {
>   		iosubmit_cmds512(portal, desc->hw, 1);
>   	} else {

Re: [PATCH v4 2/2] dmaengine: idxd: Fix incorrect descriptions for GRPCFG register

[Lijun Pan]

On 10/30/2023 9:55 PM, 'Guanjun' wrote:
> From: Guanjun <guanjun@linux.alibaba.com>
> 
> Fix incorrect descriptions for the GRPCFG register which has three
> sub-registers (GRPWQCFG, GRPENGCFG and GRPFLGCFG).
> No functional changes
> 
> Signed-off-by: Guanjun <guanjun@linux.alibaba.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com>
> ---

Acked-by: Lijun Pan <lijun.pan@intel.com>

>   drivers/dma/idxd/registers.h | 12 +++++++-----
>   1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h
> index 7b54a3939ea1..315c004f58e4 100644
> --- a/drivers/dma/idxd/registers.h
> +++ b/drivers/dma/idxd/registers.h
> @@ -440,12 +440,14 @@ union wqcfg {
>   /*
>    * This macro calculates the offset into the GRPCFG register
>    * idxd - struct idxd *
> - * n - wq id
> - * ofs - the index of the 32b dword for the config register
> + * n - group id
> + * ofs - the index of the 64b qword for the config register
>    *
> - * The WQCFG register block is divided into groups per each wq. The n index
> - * allows us to move to the register group that's for that particular wq.
> - * Each register is 32bits. The ofs gives us the number of register to access.
> + * The GRPCFG register block is divided into three sub-registers, which
> + * are GRPWQCFG, GRPENGCFG and GRPFLGCFG. The n index allows us to move
> + * to the register block that contains the three sub-registers.
> + * Each register block is 64bits. And the ofs gives us the offset
> + * within the GRPWQCFG register to access.
>    */
>   #define GRPWQCFG_OFFSET(idxd_dev, n, ofs) ((idxd_dev)->grpcfg_offset +\
>   					   (n) * GRPCFG_SIZE + sizeof(u64) * (ofs))