[Qemu-devel] [PATCH RFC 0/4] ARM SMMUv3 Emulation

[<pmallapp@broadcom.com>]

From: Prem Mallappa <pmallapp@broadcom.com>

Implementation Notes:

        - SMMUv3 model, as per ARM SMMUv3 11.0 spec
        - Works with Linux Kernel 4.4 SMMUv3 Driver By Will Deacon.
	- Stage1 only
        - only LPAE Translation tables supported
        - BE for translation tables is not supported.
        - Save/Restore not supported "YET".
        - Broadcom variant added, not much different at the moment

Untested
	- Stage2 only
	- Stage1+Stage2 support

Future planned work:
	- MSI(x) support

Test Case setup:
-------------------
 - DPDK + VFIO (DPDK from git master)
 - QEMU (master)
 - Linux Kernel 4.5
 - Bridge interface in Linux

1. Create Bridge interface
- create br0
	sudo brctl addbr br0
- create tap0 tap1 
	sudo ip tuntap add dev tap0 mode tap user ${USER}
	sudo ip tuntap add dev tap1 mode tap user ${USER}

- assign IP address to br0
	sudo ip addr add 20.40.60.80/24 dev br0

2. Launch QEMU (3 ethernet, one unused, the virtio-net is used for NFS)

     pmallapp> aarch64-softmmu/qemu-system-aarch64 -machine type=virt -cpu cortex-a57 \
		-m 4192 -smp 4 -nographic \
		-append 'console=ttyAMA0,38400n8 mem=4192M  root=/dev/nfs \
			nfsroot=10.0.2.2:/srv/nfsroot/v8,vers=3,nolock rw e1000e.IntMode=2 \
			ip=10.0.2.15:10.0.2.2:10.0.2.2:255.255.255.0:prems:eth0  ' \
		-kernel /home/pmallapp/work/vulcan/armsw/linux/build/arch/arm64/boot/Image \
		-chardev socket,id=serial0,host=localhost,port=5000,server,telnet \
		-serial chardev:serial0 \
		-chardev socket,id=mon0,host=localhost,port=6001,server,telnet,nowait \
		-monitor chardev:mon0 \
		-device pci-bridge,bus=pcie.0,multifunction=on,chassis_nr=2,msi=on,id=pcie.2 \
		-netdev user,id=eth0 \
		-device virtio-net-device,netdev=eth0 \
		-netdev tap,id=eth1,ifname=tap0,script=no,downscript=no \
		-device e1000,netdev=eth1,bus=pcie.2,addr=3 \
		-netdev tap,id=eth2,ifname=tap1,script=no,downscript=no \
		-device virtio-net-pci,netdev=eth2,vectors=0

3. Remove Device from Linux, Add to VFIO, create some reserved Huge pages
	# lspci -nn
	00:00.0 Host bridge [0600]: Red Hat, Inc. Device [1b36:0008]
	00:01.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCI-PCI bridge [1b36:0001]
	00:02.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device [1af4:1000]
	01:03.0 Ethernet controller [0200]: Intel Corporation 82540EM Gigabit Ethernet Controller [8086:100e] (rev 03)
	
	echo 0000:01:03.0 > /sys/bus/pci/devices/0000:01:03.0/driver/unbind
	echo 8086 100e > /sys/bus/pci/drivers/vfio-pci/new_id

	echo 300 > /proc/sys/vm/nr_hugepages (my case HugePageSize is 2MB)

4. Launch DPDK app, testpmd 
	./testpmd -c 0xf  -w 0000:01:03.0 -- --rxq=1 --txq= rxd=32  \
		--txd=32 --mbuf-size=32 --total-num-mbufs=1025 --max-pkt-len=512 -i
	testpmd> set fwd icmpecho
	testpmd> start

5. From host ping any address in 20.40.60.80/24 range
	ping 20.40.60.90 -I br0 -c 10 -r

Prem Mallappa (4):
  arm: smmu: ARM SMMUv3 emulation
  hw: arm: Added smmuv3 files for build
  hw: arm: Add SMMUv3 to virt platform
  devicetree: Added new APIs to make use of more fdt functions

 default-configs/aarch64-softmmu.mak |    1 +
 device_tree.c                       |   35 +
 hw/arm/Makefile.objs                |    1 +
 hw/arm/smmuv3-internal.h            |  343 ++++++++
 hw/arm/smmuv3.c                     | 1530 +++++++++++++++++++++++++++++++++++
 hw/arm/virt.c                       |   62 ++
 include/hw/arm/smmuv3.h             |   39 +
 include/hw/arm/virt.h               |    2 +
 include/sysemu/device_tree.h        |   18 +
 9 files changed, 2031 insertions(+)
 create mode 100644 hw/arm/smmuv3-internal.h
 create mode 100644 hw/arm/smmuv3.c
 create mode 100644 include/hw/arm/smmuv3.h

-- 
2.6.4