Re: [PATCH kernel 1/2] powerpc/pseries/dma: Allow swiotlb

[Thiago Jung Bauermann <bauerman@linux.ibm.com>]

Alexey Kardashevskiy <aik@ozlabs.ru> writes:

> The commit 8617a5c5bc00 ("powerpc/dma: handle iommu bypass in
> dma_iommu_ops") merged direct DMA ops into the IOMMU DMA ops allowing
> SWIOTLB as well but only for mapping; the unmapping and bouncing parts
> were left unmodified.
>
> This adds missing direct unmapping calls to .unmap_page() and .unmap_sg().
>
> This adds missing sync callbacks and directs them to the direct DMA hooks.
>
> Fixes: 8617a5c5bc00 (powerpc/dma: handle iommu bypass in dma_iommu_ops)
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>

Nice! Thanks for working on this. I have the patch at the end of this
email to get virtio-scsi-pci and virtio-blk-pci working in a secure
guest.

I applied your patch and reverted my patch and unfortunately the guest
hangs right after mounting the disk:

[    0.185659] virtio-pci 0000:00:04.0: enabling device (0100 -> 0102)
[    0.187082] virtio-pci 0000:00:04.0: ibm,query-pe-dma-windows(2026) 2000 8000000 20000000 returned 0
[    0.187497] virtio-pci 0000:00:04.0: ibm,create-pe-dma-window(2027) 2000 8000000 20000000 10 20 returned 0 (liobn = 0x80000001 startin
g addr = 8000000 0)
[    0.226654] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.227094] Non-volatile memory driver v1.3
[    0.228950] brd: module loaded
[    0.230666] loop: module loaded
[    0.230773] ipr: IBM Power RAID SCSI Device Driver version: 2.6.4 (March 14, 2017)
[    0.233323] scsi host0: Virtio SCSI HBA
[    0.235439] scsi 0:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
[    0.369009] random: fast init done
[    0.370819] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    0.371320] sd 0:0:0:0: Power-on or device reset occurred

<snip>

[    0.380378] sd 0:0:0:0: [sda] 31457280 512-byte logical blocks: (16.1 GB/15.0 GiB)
[    0.381102] sd 0:0:0:0: [sda] Write Protect is off
[    0.381195] sd 0:0:0:0: [sda] Mode Sense: 63 00 00 08
[    0.382436] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    0.383630] sd 0:0:0:0: [sda] Optimal transfer size 0 bytes < PAGE_SIZE (65536 bytes)
[    0.391562]  sda: sda1 sda2
[    0.398101] sd 0:0:0:0: [sda] Attached SCSI disk
[    0.398205] md: Waiting for all devices to be available before autodetect
[    0.398318] md: If you don't use raid, use raid=noautodetect
[    0.398515] md: Autodetecting RAID arrays.
[    0.398585] md: autorun ...
[    0.398631] md: ... autorun DONE.
[    0.403552] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
[    0.403700] VFS: Mounted root (ext4 filesystem) readonly on device 8:2.
[    0.405258] devtmpfs: mounted
[    0.406427] Freeing unused kernel memory: 4224K
[    0.406519] This architecture does not have kernel memory protection.
[    0.406633] Run /sbin/init as init process

Sorry, I don't have any information on where the guest is stuck. I tried
<sysrq>+l, <sysrq>+t and <sysrq>+w but nothing out of the ordinary
showed up. Will try something else later.

--
Thiago Jung Bauermann
IBM Linux Technology Center



From 70d2fba809119ae2d35c9ca4269405bb5c28413a Mon Sep 17 00:00:00 2001
From: Thiago Jung Bauermann <bauerman@linux.ibm.com>
Date: Thu, 24 Jan 2019 22:40:16 -0200
Subject: [PATCH 1/1] powerpc/pseries/iommu: Don't use dma_iommu_ops on secure
 guests

Secure guest memory is inacessible to devices so regular DMA isn't
possible.

In that case set devices' dma_map_ops to NULL so that the generic
DMA code path will use SWIOTLB and DMA to bounce buffers.

Signed-off-by: Thiago Jung Bauermann <bauerman@linux.ibm.com>
---
 arch/powerpc/platforms/pseries/iommu.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 36eb1ddbac69..1636306007eb 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -50,6 +50,7 @@
 #include <asm/udbg.h>
 #include <asm/mmzone.h>
 #include <asm/plpar_wrappers.h>
+#include <asm/svm.h>

 #include "pseries.h"

@@ -1335,7 +1336,10 @@ void iommu_init_early_pSeries(void)
 	of_reconfig_notifier_register(&iommu_reconfig_nb);
 	register_memory_notifier(&iommu_mem_nb);

-	set_pci_dma_ops(&dma_iommu_ops);
+	if (is_secure_guest())
+		set_pci_dma_ops(NULL);
+	else
+		set_pci_dma_ops(&dma_iommu_ops);
 }

 static int __init disable_multitce(char *str)