From mboxrd@z Thu Jan 1 00:00:00 1970 From: Suravee Suthikulpanit Subject: Re: AMD Ryzen KVM/NPT/IOMMU issue Date: Thu, 29 Jun 2017 00:23:20 +0700 Message-ID: <545f19a3-4923-cdec-4ce9-2a4155a04f6a@amd.com> References: <575f8fbc-0fdc-f336-e3da-53f27da4b2e1@markju.com> <20170505112706.7785948c@t450s.home> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Nick Sarnie , Alex Williamson Cc: Paolo Bonzini , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Matthias Ehrenfeuchter , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , John.Bridgman-5C7GfCeVMHo@public.gmane.org List-Id: iommu@lists.linux-foundation.org On 6/25/17 12:55, Nick Sarnie wrote: > On Fri, May 5, 2017 at 1:27 PM, Alex Williamson > wrote: >> On Wed, 3 May 2017 12:28:35 -0400 >> Nick Sarnie wrote: >> >>> On Wed, May 3, 2017 at 10:37 AM, Matthias Ehrenfeuchter wrote: >>>> Hi, >>>> >>>> There are a lot of messages/threads out there about bad performance while >>>> using AMDs Ryzen with KVM GPU passthrough. It revolves all on >>>> enabling/disabling npt, while enabled overall VM performance is nice but the >>>> GPU performance gives me about 20% (and a lot of drops to zero GPU usage, >>>> while CPU/Disk/Ram also doing nothing) compared to npt disabled. But while >>>> npt is disabled overall VM performance is like beeing on 4x86 with floppy >>>> disk as only storage. (Ex. it takes 2 seconds just to open startmenu while >>>> host and vm are in idle, and neither CPU pinning, changing CPU model, >>>> changing storage device nor using hugepages changed anything). >>>> >>>> So everything I read pointed to a bug in the npt implementation? Anything I >>>> could do to get closer to the "thing" issuing this? >>>> >>>> Best Regards >>>> >>>> efeu >>>> _______________________________________________ >>>> iommu mailing list >>>> iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/iommu >>> >>> I heard from Joerg that it might be related to a lower intercept rate >>> being used when NPT is enabled, but we haven't been able to find a way >>> to trace that to confirm. >> >> Joerg/Paolo, any ideas how we might debug this? Anyone from AMD >> watching? Thanks, >> >> Alex > > > Hi all, > > A somewhat major update. > > I managed to install Xen with my GPU passthrough config and test the > performance with NPT enabled. > > There is no performance drop with NPT on Xen, it matches the GPU > performance of KVM with NPT disabled. The CPU performance is also > great. > > John Bridgman (ccd) from AMD says he's going to ask around AMD about > this next week, but it would be even better if some AMD guys that read > this ML shared their ideas or took a look. > > Let me know if you need any more information. > > Thanks, > Sarnex > _______________________________________________ > iommu mailing list > iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu > So, I'm trying to reproduce this issue on the Ryzen system w/ the following setup: * Host kernel v4.11 (with this patch https://lkml.org/lkml/2017/6/23/295) * guest VM RHEL7.3 * guest graphic driver = radeon * qemu-system-x86_64 --version QEMU emulator version 2.9.50 (v2.9.0-1659-g577caa2-dirty) * kvm-amd npt=1 * dGPU is 08:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tobago PRO [Radeon R7 360 / R9 360 OEM] (rev 81) * qemu-system-x86_64 -smp 4 -enable-kvm -M q35 -m 4096 -cpu host -bios /usr/share/qemu/bios.bin -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 -drive file=/sandbox/vm-images/rhel7.3.qcow2,if=virtio,id=disk0 -net none -vga none -nodefaults -device vfio-pci,host=08:00.0,x-vga=on,addr=0.0,multifunction=on,bus=root.1,romfile=/sandbox/vm-images/vbios.rom -usb -device usb-host,hostbus=3,hostport=1 -device usb-host,hostbus=3,hostport=3 -device vfio-pci,host=0000:08:00.1 -device vfio-pci,host=0000:09:00.0 With this setup, I am able to pass-through the dGPU and run the following test: * Starting up the guest w/ full GNOME GUI on the attached monitor. * glxgears (running @ 60 FPS) * Playing 1080p HD video on Youtube I am not noticing issues here. What kind of test are you running in the guest VM? Thanks, Suravee