public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2] net/mlx4_core: VF probe fail when HW support 64-bit coherent DMA
@ 2017-01-09  9:38 Shamir Rabinovitch
       [not found] ` <1483954699-17826-1-git-send-email-shamir.rabinovitch-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
  0 siblings, 1 reply; 6+ messages in thread
From: Shamir Rabinovitch @ 2017-01-09  9:38 UTC (permalink / raw)
  To: linux-rdma-u79uwXL29TY76Z2rM5mHXA
  Cc: yishaih-VPRAkNaXOzVWk0Htik3J/w,
	shamir.rabinovitch-QHcLZuEGTsvQT0dZR+AlfA

If IOMMU support 64-bit coherent DMA mlx4_core driver will try to use it
and VF probe will fail due to firmware error.

Force all mlx4_core VFs coherent DMA to 32-bit only!

Kernel log when issue occur:

[1383654.766249] mlx4_core 0006:01:00.1: Sending reset
[1383654.775971] mlx4_core 0006:01:00.0: Received reset from slave:1
[1383654.788087] mlx4_core 0006:01:00.1: Sending vhcr0
[1383664.318338] mlx4_core 0006:01:00.0: command 0x2e failed: fw status = 0x1
[1383664.318342] mlx4_core 0006:01:00.0: mlx4_master_process_vhcr: Failed
reading vhcr ret: 0xfffffffb
[1383664.318345] mlx4_core 0006:01:00.0: Failed processing vhcr for slave:1,
resetting slave
[1383664.318352] mlx4_core 0006:01:00.0: Turn on internal error to force
reset, slave=1, cmd=0x5
[1383664.318415] mlx4_core 0006:01:00.0: slave:1 is out of sync, cmd=0x5,
last command=0x0, reset is needed
[1383664.318418] mlx4_core 0006:01:00.0: Turn on internal error to force
reset, slave=1, cmd=0x5
[1383664.318501] mlx4_core 0006:01:00.0: slave:1 is out of sync, cmd=0x5,
last command=0x0, reset is needed
[1383664.318504] mlx4_core 0006:01:00.0: Turn on internal error to force
reset, slave=1, cmd=0x5
[1383664.318513] mlx4_core 0006:01:00.1: HCA minimum page size:1
[1383664.318515] mlx4_core 0006:01:00.1: UAR size:4096 != kernel PAGE_SIZE of
8192
[1383664.318517] mlx4_core 0006:01:00.1: Failed to obtain slave caps

Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>

---

Changelog:

v1 -> v2:
Review comments from Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>.
Verified and only VF require 32-bit coherent DMA.
PF can still use 64-bit coherent DMA.

---

---
 drivers/net/ethernet/mellanox/mlx4/main.c |   20 +++++++++++++++++---
 1 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index bffa6f3..131cbc9 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -3719,9 +3719,23 @@ static int __mlx4_init_one(struct pci_dev *pdev, int pci_dev_data,
 			goto err_release_regions;
 		}
 	}
-	err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
-	if (err) {
-		dev_warn(&pdev->dev, "Warning: couldn't set 64-bit consistent PCI DMA mask\n");
+	if (!(pci_dev_data & MLX4_PCI_DEV_IS_VF)) {
+		err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+		if (err) {
+			dev_warn(&pdev->dev,
+				 "Warning: couldn't set 64-bit consistent PCI DMA mask\n");
+			err = pci_set_consistent_dma_mask(pdev,
+							  DMA_BIT_MASK(32));
+			if (err) {
+				dev_err(&pdev->dev,
+					"Can't set consistent PCI DMA mask, aborting\n");
+				goto err_release_regions;
+			}
+		}
+	} else {
+		/* CX3 firmware 2.11.1280 does not support 64-bit coherent
+		 * DMA for VFs.
+		 */
 		err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
 		if (err) {
 			dev_err(&pdev->dev, "Can't set consistent PCI DMA mask, aborting\n");
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-01-25 12:42 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-01-09  9:38 [PATCH v2] net/mlx4_core: VF probe fail when HW support 64-bit coherent DMA Shamir Rabinovitch
     [not found] ` <1483954699-17826-1-git-send-email-shamir.rabinovitch-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-01-09 10:21   ` Leon Romanovsky
     [not found]     ` <20170109102148.GY15685-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-01-09 11:16       ` Shamir Rabinovitch
2017-01-11  8:53   ` Leon Romanovsky
     [not found]     ` <20170111085358.GX7218-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-01-24 21:34       ` Doug Ledford
     [not found]         ` <1485293674.43764.86.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2017-01-25 12:42           ` Shamir Rabinovitch

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox