* [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os @ 2020-10-16 15:19 Stefan Fritsch 2020-10-19 9:19 ` Zhenyu Wang 0 siblings, 1 reply; 6+ messages in thread From: Stefan Fritsch @ 2020-10-16 15:19 UTC (permalink / raw) To: intel-gfx Hi, if Linux is running as a guest and the host is doing igd-pass-thorugh with VT-d enabled, the i915 driver does not work all that great. The most obvious problem is that there are dozens of 'Fault errors on pipe A' errrors logged per second, but depending on the hardware there can be other issues, too. I will send a patch to rate-limit that message in a separate mail. The i915 has various quirks for VT-d and these should be enabled even if Linux is running as a guest and does itself have iommu enabled. I have checked that making intel_vtd_active() form i915_drv.h return true makes the error messages go away. How could Linux detect this situation? Maybe simply check the Hypervisor cpuid bit? Or would you prefer a module parameter, or a combination of both? Or is there another way to detect that VT-d is enabled for the igd device? Cheers, Stefan _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os 2020-10-16 15:19 [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os Stefan Fritsch @ 2020-10-19 9:19 ` Zhenyu Wang 2020-10-19 9:57 ` Chris Wilson 0 siblings, 1 reply; 6+ messages in thread From: Zhenyu Wang @ 2020-10-19 9:19 UTC (permalink / raw) To: Stefan Fritsch; +Cc: intel-gfx, intel-gvt-dev [-- Attachment #1.1: Type: text/plain, Size: 1376 bytes --] On 2020.10.16 17:19:19 +0200, Stefan Fritsch wrote: > Hi, > > if Linux is running as a guest and the host is doing igd-pass-thorugh with > VT-d enabled, the i915 driver does not work all that great. The most > obvious problem is that there are dozens of 'Fault errors on pipe A' > errrors logged per second, but depending on the hardware there can be > other issues, too. I will send a patch to rate-limit that message in a > separate mail. > > The i915 has various quirks for VT-d and these should be enabled even if > Linux is running as a guest and does itself have iommu enabled. I have > checked that making intel_vtd_active() form i915_drv.h return true makes > the error messages go away. How could Linux detect this situation? Maybe > simply check the Hypervisor cpuid bit? Or would you prefer a module > parameter, or a combination of both? Or is there another way to detect > that VT-d is enabled for the igd device? > I think that's right, although I haven't tried to force intel_vtd_active() for guest, but I did see those fault errors on some machine. You can use hypervisor cpuid bit, and need to seperate case for GVT which is detected by intel_vgpu_active(), but I'm not sure if this should be taken in nested case, suppose those quirks should still work? -- $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827 [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 160 bytes --] _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os 2020-10-19 9:19 ` Zhenyu Wang @ 2020-10-19 9:57 ` Chris Wilson 2020-10-19 9:51 ` Zhenyu Wang 0 siblings, 1 reply; 6+ messages in thread From: Chris Wilson @ 2020-10-19 9:57 UTC (permalink / raw) To: Stefan Fritsch, Zhenyu Wang; +Cc: intel-gfx, intel-gvt-dev Quoting Zhenyu Wang (2020-10-19 10:19:09) > On 2020.10.16 17:19:19 +0200, Stefan Fritsch wrote: > > Hi, > > > > if Linux is running as a guest and the host is doing igd-pass-thorugh with > > VT-d enabled, the i915 driver does not work all that great. The most > > obvious problem is that there are dozens of 'Fault errors on pipe A' > > errrors logged per second, but depending on the hardware there can be > > other issues, too. I will send a patch to rate-limit that message in a > > separate mail. > > > > The i915 has various quirks for VT-d and these should be enabled even if > > Linux is running as a guest and does itself have iommu enabled. I have > > checked that making intel_vtd_active() form i915_drv.h return true makes > > the error messages go away. How could Linux detect this situation? Maybe > > simply check the Hypervisor cpuid bit? Or would you prefer a module > > parameter, or a combination of both? Or is there another way to detect > > that VT-d is enabled for the igd device? > > > > I think that's right, although I haven't tried to force intel_vtd_active() > for guest, but I did see those fault errors on some machine. You can use > hypervisor cpuid bit, and need to seperate case for GVT which is detected by > intel_vgpu_active(), but I'm not sure if this should be taken in nested case, > suppose those quirks should still work? Do we need it for gvt since the guest has no access to HW, so the host should be doing all the vt'd w/a. (In particular, the scanout overfetch causing the problems here.) E.g. in gvt, the guest framebuffer is transported via magics to the host, and the host creates a GGTT entry for it. For detecting a hypervisor, following on from the KVM_CPUID_SIGNATURE hint, we find arch/x86 does detect_hypervisor_platform() at load which populates EXPORT_SYMBOL(x86_hyper_type) So something like, #include <asm/hypervisor.h> @@ -1764,7 +1766,7 @@ static inline bool intel_vtd_active(void) if (intel_iommu_gfx_mapped) return true; #endif - return false; + return x86_hyper_type != X86_HYPER_NATIVE; } should work. -Chris _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os 2020-10-19 9:57 ` Chris Wilson @ 2020-10-19 9:51 ` Zhenyu Wang 2020-10-19 10:20 ` Chris Wilson 0 siblings, 1 reply; 6+ messages in thread From: Zhenyu Wang @ 2020-10-19 9:51 UTC (permalink / raw) To: Chris Wilson; +Cc: intel-gvt-dev, intel-gfx, Stefan Fritsch [-- Attachment #1.1: Type: text/plain, Size: 2752 bytes --] On 2020.10.19 10:57:38 +0100, Chris Wilson wrote: > Quoting Zhenyu Wang (2020-10-19 10:19:09) > > On 2020.10.16 17:19:19 +0200, Stefan Fritsch wrote: > > > Hi, > > > > > > if Linux is running as a guest and the host is doing igd-pass-thorugh with > > > VT-d enabled, the i915 driver does not work all that great. The most > > > obvious problem is that there are dozens of 'Fault errors on pipe A' > > > errrors logged per second, but depending on the hardware there can be > > > other issues, too. I will send a patch to rate-limit that message in a > > > separate mail. > > > > > > The i915 has various quirks for VT-d and these should be enabled even if > > > Linux is running as a guest and does itself have iommu enabled. I have > > > checked that making intel_vtd_active() form i915_drv.h return true makes > > > the error messages go away. How could Linux detect this situation? Maybe > > > simply check the Hypervisor cpuid bit? Or would you prefer a module > > > parameter, or a combination of both? Or is there another way to detect > > > that VT-d is enabled for the igd device? > > > > > > > I think that's right, although I haven't tried to force intel_vtd_active() > > for guest, but I did see those fault errors on some machine. You can use > > hypervisor cpuid bit, and need to seperate case for GVT which is detected by > > intel_vgpu_active(), but I'm not sure if this should be taken in nested case, > > suppose those quirks should still work? > > Do we need it for gvt since the guest has no access to HW, so the host > should be doing all the vt'd w/a. (In particular, the scanout overfetch > causing the problems here.) E.g. in gvt, the guest framebuffer is > transported via magics to the host, and the host creates a GGTT entry > for it. GVT doesn't require vt-d at all. Current gvt display is fully virtualized, so if by any way that map guest framebuffer directly on host display, it should still be handled by i915 with possible vt-d workaround for alignment. And looks some other vt-d w/a just brings unnecessary actions for gvt guest. So I think we should stick with real vt-d case. > > For detecting a hypervisor, following on from the KVM_CPUID_SIGNATURE > hint, we find arch/x86 does detect_hypervisor_platform() at load which > populates EXPORT_SYMBOL(x86_hyper_type) > > So something like, > > #include <asm/hypervisor.h> > @@ -1764,7 +1766,7 @@ static inline bool intel_vtd_active(void) > if (intel_iommu_gfx_mapped) > return true; > #endif > - return false; > + return x86_hyper_type != X86_HYPER_NATIVE; > } > > should work. > -Chris -- $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827 [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 160 bytes --] _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os 2020-10-19 9:51 ` Zhenyu Wang @ 2020-10-19 10:20 ` Chris Wilson 2020-10-19 10:07 ` Zhenyu Wang 0 siblings, 1 reply; 6+ messages in thread From: Chris Wilson @ 2020-10-19 10:20 UTC (permalink / raw) To: Zhenyu Wang; +Cc: intel-gvt-dev, intel-gfx, Stefan Fritsch Quoting Zhenyu Wang (2020-10-19 10:51:44) > On 2020.10.19 10:57:38 +0100, Chris Wilson wrote: > > Quoting Zhenyu Wang (2020-10-19 10:19:09) > > > On 2020.10.16 17:19:19 +0200, Stefan Fritsch wrote: > > > > Hi, > > > > > > > > if Linux is running as a guest and the host is doing igd-pass-thorugh with > > > > VT-d enabled, the i915 driver does not work all that great. The most > > > > obvious problem is that there are dozens of 'Fault errors on pipe A' > > > > errrors logged per second, but depending on the hardware there can be > > > > other issues, too. I will send a patch to rate-limit that message in a > > > > separate mail. > > > > > > > > The i915 has various quirks for VT-d and these should be enabled even if > > > > Linux is running as a guest and does itself have iommu enabled. I have > > > > checked that making intel_vtd_active() form i915_drv.h return true makes > > > > the error messages go away. How could Linux detect this situation? Maybe > > > > simply check the Hypervisor cpuid bit? Or would you prefer a module > > > > parameter, or a combination of both? Or is there another way to detect > > > > that VT-d is enabled for the igd device? > > > > > > > > > > I think that's right, although I haven't tried to force intel_vtd_active() > > > for guest, but I did see those fault errors on some machine. You can use > > > hypervisor cpuid bit, and need to seperate case for GVT which is detected by > > > intel_vgpu_active(), but I'm not sure if this should be taken in nested case, > > > suppose those quirks should still work? > > > > Do we need it for gvt since the guest has no access to HW, so the host > > should be doing all the vt'd w/a. (In particular, the scanout overfetch > > causing the problems here.) E.g. in gvt, the guest framebuffer is > > transported via magics to the host, and the host creates a GGTT entry > > for it. > > GVT doesn't require vt-d at all. Current gvt display is fully virtualized, > so if by any way that map guest framebuffer directly on host display, it > should still be handled by i915 with possible vt-d workaround for alignment. > And looks some other vt-d w/a just brings unnecessary actions for gvt guest. > So I think we should stick with real vt-d case. It's not too bad; we should only be applying the vtd workaround on interacting with the HW. So for gvt running under a kvm hypervisor, setting intel_vtd_active() should not impact us at all... -Chris _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os 2020-10-19 10:20 ` Chris Wilson @ 2020-10-19 10:07 ` Zhenyu Wang 0 siblings, 0 replies; 6+ messages in thread From: Zhenyu Wang @ 2020-10-19 10:07 UTC (permalink / raw) To: Chris Wilson; +Cc: intel-gvt-dev, intel-gfx, Stefan Fritsch [-- Attachment #1.1: Type: text/plain, Size: 2801 bytes --] On 2020.10.19 11:20:40 +0100, Chris Wilson wrote: > Quoting Zhenyu Wang (2020-10-19 10:51:44) > > On 2020.10.19 10:57:38 +0100, Chris Wilson wrote: > > > Quoting Zhenyu Wang (2020-10-19 10:19:09) > > > > On 2020.10.16 17:19:19 +0200, Stefan Fritsch wrote: > > > > > Hi, > > > > > > > > > > if Linux is running as a guest and the host is doing igd-pass-thorugh with > > > > > VT-d enabled, the i915 driver does not work all that great. The most > > > > > obvious problem is that there are dozens of 'Fault errors on pipe A' > > > > > errrors logged per second, but depending on the hardware there can be > > > > > other issues, too. I will send a patch to rate-limit that message in a > > > > > separate mail. > > > > > > > > > > The i915 has various quirks for VT-d and these should be enabled even if > > > > > Linux is running as a guest and does itself have iommu enabled. I have > > > > > checked that making intel_vtd_active() form i915_drv.h return true makes > > > > > the error messages go away. How could Linux detect this situation? Maybe > > > > > simply check the Hypervisor cpuid bit? Or would you prefer a module > > > > > parameter, or a combination of both? Or is there another way to detect > > > > > that VT-d is enabled for the igd device? > > > > > > > > > > > > > I think that's right, although I haven't tried to force intel_vtd_active() > > > > for guest, but I did see those fault errors on some machine. You can use > > > > hypervisor cpuid bit, and need to seperate case for GVT which is detected by > > > > intel_vgpu_active(), but I'm not sure if this should be taken in nested case, > > > > suppose those quirks should still work? > > > > > > Do we need it for gvt since the guest has no access to HW, so the host > > > should be doing all the vt'd w/a. (In particular, the scanout overfetch > > > causing the problems here.) E.g. in gvt, the guest framebuffer is > > > transported via magics to the host, and the host creates a GGTT entry > > > for it. > > > > GVT doesn't require vt-d at all. Current gvt display is fully virtualized, > > so if by any way that map guest framebuffer directly on host display, it > > should still be handled by i915 with possible vt-d workaround for alignment. > > And looks some other vt-d w/a just brings unnecessary actions for gvt guest. > > So I think we should stick with real vt-d case. > > It's not too bad; we should only be applying the vtd workaround on > interacting with the HW. So for gvt running under a kvm hypervisor, > setting intel_vtd_active() should not impact us at all... Current quick view seems that gen8_ggtt_clear_range is unnecessary, but maybe not a big deal. Others seem ok... -- $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827 [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 160 bytes --] _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-10-19 10:28 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-10-16 15:19 [Intel-gfx] drm/i915: Detecting Vt-d when running as guest os Stefan Fritsch 2020-10-19 9:19 ` Zhenyu Wang 2020-10-19 9:57 ` Chris Wilson 2020-10-19 9:51 ` Zhenyu Wang 2020-10-19 10:20 ` Chris Wilson 2020-10-19 10:07 ` Zhenyu Wang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox