* Re: Local Display Direct Flip Feature Discussion
[not found] ` <20181220113335.3lldza37rfwb462h@sirius.home.kraxel.org>
@ 2018-12-21 2:24 ` Zhang, Tina
2018-12-21 2:52 ` Zhenyu Wang
1 sibling, 0 replies; 3+ messages in thread
From: Zhang, Tina @ 2018-12-21 2:24 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Wang, Zhenyu Z, intel-gfx@lists.freedesktop.org,
dri-devel@lists.freedesktop.org, Yuan, Hang, Lv, Zhiyuan,
Vetter, Daniel, Wang, Hongbo, intel-gvt-dev
+ intel-gfx and dri-devel
Hi,
This is the Gvt-g local display direct flip discussion. And please reference here (https://lists.freedesktop.org/archives/intel-gvt-dev/2018-December/004594.html) to see what the feature is.
Mostly, with Gvt-g local display direct flip feature, vGPU can directly post its framebuffer to the assigned local HW display plane w/o host userspace involvement. We propose to use drm_framebuffer to stand for the framebuffer attached on a vGPU's plane. Since GVT-g can guarantee this drm_framebuffer is updated with the latest info of the framebuffer attached on the vGPU's plane during guest page flip, atomic async plane update mechanism is used in i915 to help updating the vGPU assigned HW planes efficiently with those special drm_framebuffers.
Host userspace is in charge of the HW plane assignment. And currently we are discussing about:
1) Whether the drm_framebuffer can be generic enough to deal with the direct flip use cases?
In our case, the drm_framebuffer has no gem backends, as the display memory management is handled by guest.
2) How to expose the drm_framebuffer to the host userspace?
a) Through vfio/display ioctl
The kernel implementation is easy. But this might turn qemu into a display manager. Qemu has to handle all the KMS mode-setting stuff things.
b) Through drm/i915 ioctl
This needs some code change in kernel space to deal with this ioctl between i915 and GVT-g. But the good thing is this ioctl can be used by the userspace display manager w/o vfio/display's help.
Thanks.
> -----Original Message-----
> From: Gerd Hoffmann [mailto:kraxel@redhat.com]
> Sent: Thursday, December 20, 2018 7:34 PM
> To: Zhang, Tina <tina.zhang@intel.com>
> Cc: Tian, Kevin <kevin.tian@intel.com>; Wang, Zhenyu Z
> <zhenyu.z.wang@intel.com>; Wang, Zhi A <zhi.a.wang@intel.com>; He, Min
> <min.he@intel.com>; Yuan, Hang <hang.yuan@intel.com>; Alex Williamson
> <alex.williamson@redhat.com>; Lv, Zhiyuan <zhiyuan.lv@intel.com>; Vetter,
> Daniel <daniel.vetter@intel.com>; intel-gvt-dev <intel-gvt-
> dev@lists.freedesktop.org>; Wang, Hongbo <hongbo.wang@intel.com>
> Subject: Re: Local Display Direct Flip Feature Discussion
>
> On Thu, Dec 20, 2018 at 08:45:09AM +0000, Zhang, Tina wrote:
> >
> >
> > > -----Original Message-----
> > > From: intel-gvt-dev
> > > [mailto:intel-gvt-dev-bounces@lists.freedesktop.org] On Behalf Of
> > > Gerd Hoffmann
> > > Sent: Wednesday, December 19, 2018 7:26 PM
> > > To: Zhang, Tina <tina.zhang@intel.com>
> > > Cc: Tian, Kevin <kevin.tian@intel.com>; Wang, Zhenyu Z
> > > <zhenyu.z.wang@intel.com>; Wang, Zhi A <zhi.a.wang@intel.com>; He,
> > > Min <min.he@intel.com>; Yuan, Hang <hang.yuan@intel.com>; Alex
> > > Williamson <alex.williamson@redhat.com>; Lv, Zhiyuan
> > > <zhiyuan.lv@intel.com>; Vetter, Daniel <daniel.vetter@intel.com>;
> > > intel-gvt-dev <intel-gvt- dev@lists.freedesktop.org>; Wang, Hongbo
> > > <hongbo.wang@intel.com>
> > > Subject: Re: Local Display Direct Flip Feature Discussion
> > >
> > > Hi,
> > >
> > > > > Isn't a framebuffer just a gem object with metadata (fourcc,
> > > > > width, height, stride, ...) attached? So I'm wondering how that
> > > > > works in detail. What happens on page-flip? Do you make the
> > > > > framebuffer reference another gem object then? Or do you blit
> > > > > the guest display to the framebuffer?
> > > > The special DRM framebuffer is transparent to the guest display driver.
> > > > We just want to use this object to save the guest framebuffer info
> > > > in host-side.
> > >
> > > Hmm, so this object isn't a normal drm framebuffer. You are just
> > > masquerading it as drm framebuffer, so you can assign it to a drm
> > > crtc or drm plane.
> > Not so bad actually :-)
> > Host just wants to use a drm framebuffer to describe the drm
> > framebuffer attached on a vGPU's plane. And the only special thing is
> > that this host drm frambuffer has on gem backends, which means host
> > cannot manage the
>
> ... has no gem ... ?
>
> > memory of this drm framebuffer. And it's OK, as the guest does the
> management.
> > Besides, generic drm framebuffer was designed to be able to deal with
> > this kind of situation.
>
> Hmm, ok. Never dealed with framebuffers not backed by gem objects so far,
> seems to not be very common too. But the doc comments in
> include/drm/drm_framebuffer.h suggest it is fine indeed.
>
> What is the plan for hardware cursor support? Support two framebuffers, for
> primary and cursor?
We got the hardware cursor support in our to-go list:
https://lists.freedesktop.org/archives/intel-gvt-dev/2018-December/004559.html
We will support both primary and cursor.
>
> > > It makes sense to allow mapping guest outputs to host outputs. I
> > > think it is more useful to handle that at crtc level not plane
> > > level, so it'll work for both primary and cursor plane. I think it
> > > would be cleaner to introduce new drm (generic or i915) APIs for
> > > that instead of creating special framebuffer objects which behave in non-
> standard ways.
> > The APIs solution is one of our options and we have patch for it. The
> > userspace interface is still under discussion. And here are the three
> candidates:
> > 1) Through i915 ioctl
> > 2) Through vfio/display ioctl
> > 3) Through GVT-g sys fs or debugfs
> > And option2 is considered as the preferred one, as this drm
> > framebuffer is related to the vGPU.
>
> (3) Having this (in debugfs) would be useful for debugging purposes,
> even in addition to (1) or (2).
>
> (2) Implies qemu must handle it, so support must either be implemented
> in qemu directly, or in another process cooperating with qemu in
> some way (extending spice protocol & spice client comes to mind).
> Given that the input side (mouse and kbd events) need cooperation
> with qemu anyway this might not be much of a limitation though.
>
> (1) Would allow to handle this without having to worry about qemu/vfio
> at all. Needs some infrastructure (drm ioctl to enumerate vgpus for
> example). Possibly the amdgpu guys (which are doing vgpu using
> sr/iov instead of mdev) are interested in this too.
>
> No clear winner, but I tend to agree that (2) looks best.
>
> > > Alternatively try to tackle the problem in a completely different way.
> > > Right now qemu will check for display changes using a timer. We
> > > could add some page flip notification mechanism (using an eventfd
> > > for example) so qemu can update the host plane (or notify spice
> > > client) instantly instead of waiting for the next display refresh timer tick.
> > We could do that. But the local display direct flip feature doesn't
> > want to involve the host userspace, considering the performance.
>
> Is the additional switch to qemu actually that much of a performance
> problem? I somehow doubt it, it's not like we are doing hundreds of page-
> flips per second.
If we only focus on the vblank events (not page flip events), I think it might be
OK, as the vblank event is per crtc not per plane. I think we can use dma-buf
to have a try.
Thanks.
BR,
Tina
>
> > Maybe we can add this mechanism to dma-buf use case.
>
> That would be cool.
>
> cheers,
> Gerd
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: Local Display Direct Flip Feature Discussion
[not found] ` <20181220113335.3lldza37rfwb462h@sirius.home.kraxel.org>
2018-12-21 2:24 ` Local Display Direct Flip Feature Discussion Zhang, Tina
@ 2018-12-21 2:52 ` Zhenyu Wang
2018-12-21 13:48 ` Daniel Vetter
1 sibling, 1 reply; 3+ messages in thread
From: Zhenyu Wang @ 2018-12-21 2:52 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: intel-gfx, dri-devel, Yuan, Hang, Lv, Zhiyuan, Vetter, Daniel,
Wang, Hongbo, intel-gvt-dev
[-- Attachment #1.1: Type: text/plain, Size: 5694 bytes --]
On 2018.12.20 12:33:35 +0100, Gerd Hoffmann wrote:
> On Thu, Dec 20, 2018 at 08:45:09AM +0000, Zhang, Tina wrote:
> >
> >
> > > -----Original Message-----
> > > From: intel-gvt-dev [mailto:intel-gvt-dev-bounces@lists.freedesktop.org] On
> > > Behalf Of Gerd Hoffmann
> > > Sent: Wednesday, December 19, 2018 7:26 PM
> > > To: Zhang, Tina <tina.zhang@intel.com>
> > > Cc: Tian, Kevin <kevin.tian@intel.com>; Wang, Zhenyu Z
> > > <zhenyu.z.wang@intel.com>; Wang, Zhi A <zhi.a.wang@intel.com>; He, Min
> > > <min.he@intel.com>; Yuan, Hang <hang.yuan@intel.com>; Alex Williamson
> > > <alex.williamson@redhat.com>; Lv, Zhiyuan <zhiyuan.lv@intel.com>; Vetter,
> > > Daniel <daniel.vetter@intel.com>; intel-gvt-dev <intel-gvt-
> > > dev@lists.freedesktop.org>; Wang, Hongbo <hongbo.wang@intel.com>
> > > Subject: Re: Local Display Direct Flip Feature Discussion
> > >
> > > Hi,
> > >
> > > > > Isn't a framebuffer just a gem object with metadata (fourcc, width,
> > > > > height, stride, ...) attached? So I'm wondering how that works in
> > > > > detail. What happens on page-flip? Do you make the framebuffer
> > > > > reference another gem object then? Or do you blit the guest display
> > > > > to the framebuffer?
> > > > The special DRM framebuffer is transparent to the guest display driver.
> > > > We just want to use this object to save the guest framebuffer info in
> > > > host-side.
> > >
> > > Hmm, so this object isn't a normal drm framebuffer. You are just
> > > masquerading it as drm framebuffer, so you can assign it to a drm crtc or drm
> > > plane.
> > Not so bad actually :-)
> > Host just wants to use a drm framebuffer to describe the drm framebuffer
> > attached on a vGPU's plane. And the only special thing is that this host drm
> > frambuffer has on gem backends, which means host cannot manage the
>
> ... has no gem ... ?
>
> > memory of this drm framebuffer. And it's OK, as the guest does the management.
> > Besides, generic drm framebuffer was designed to be able to deal with this kind
> > of situation.
>
> Hmm, ok. Never dealed with framebuffers not backed by gem objects so
> far, seems to not be very common too. But the doc comments in
> include/drm/drm_framebuffer.h suggest it is fine indeed.
>
> What is the plan for hardware cursor support? Support two framebuffers,
> for primary and cursor?
>
> > > It makes sense to allow mapping guest outputs to host outputs. I think it is
> > > more useful to handle that at crtc level not plane level, so it'll work for both
> > > primary and cursor plane. I think it would be cleaner to introduce new drm
> > > (generic or i915) APIs for that instead of creating special framebuffer objects
> > > which behave in non-standard ways.
> > The APIs solution is one of our options and we have patch for it. The userspace
> > interface is still under discussion. And here are the three candidates:
> > 1) Through i915 ioctl
> > 2) Through vfio/display ioctl
> > 3) Through GVT-g sys fs or debugfs
> > And option2 is considered as the preferred one, as this drm framebuffer is related
> > to the vGPU.
>
> (3) Having this (in debugfs) would be useful for debugging purposes,
> even in addition to (1) or (2).
>
> (2) Implies qemu must handle it, so support must either be implemented
> in qemu directly, or in another process cooperating with qemu in
> some way (extending spice protocol & spice client comes to mind).
> Given that the input side (mouse and kbd events) need cooperation
> with qemu anyway this might not be much of a limitation though.
>
> (1) Would allow to handle this without having to worry about qemu/vfio
> at all. Needs some infrastructure (drm ioctl to enumerate vgpus for
> example). Possibly the amdgpu guys (which are doing vgpu using
> sr/iov instead of mdev) are interested in this too.
>
> No clear winner, but I tend to agree that (2) looks best.
>
yeah, that's always my idea on this, because vfio interface would be
only place to align with vGPU/mdev life cycle. GVT specific sysfs should
only contain kind of static configuration or things won't depend on vGPU
open or close, so isn't a good place. Debugfs could be used for anything
helpful to dump.
Original idea is still using dmabuf object to present vGPU plane.
Instead of using polling for now as in qemu drm/kms display POC code,
as you sugguested, some notification method could be created for vGPU
plane flip.
For direct flip, some flag or attribute could be added for drm/i915
ioctl when create drm_framebuffer from dmabuf or when modesetting to
say "this is special vGPU plane buffer so pls help to do atomic async
update on it when guest flip".
> > > Alternatively try to tackle the problem in a completely different way.
> > > Right now qemu will check for display changes using a timer. We could add
> > > some page flip notification mechanism (using an eventfd for example) so
> > > qemu can update the host plane (or notify spice client) instantly instead of
> > > waiting for the next display refresh timer tick.
> > We could do that. But the local display direct flip feature doesn't want to involve
> > the host userspace, considering the performance.
>
> Is the additional switch to qemu actually that much of a performance
> problem? I somehow doubt it, it's not like we are doing hundreds of
> page-flips per second.
>
> > Maybe we can add this mechanism to dma-buf use case.
>
> That would be cool.
>
> cheers,
> Gerd
>
--
Open Source Technology Center, Intel ltd.
$gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 3+ messages in thread