Ask for comments of getting guest framebuffer in igvt-g

Ask for comments of getting guest framebuffer in igvt-g

[Zhiyuan Lv]

Dear i915 developers,

Here I have one topic hoping to get your comments and suggestions.
Basically it is about graphics virtualization(igvt-g), for the purpose
of host system to get virtual machine's framebuffer. We would like to
hear your opinions about some design opens. Below is the
patch and some more detailed description. We appreciate your time
on that, and thanks in advance for any comments!

https://patchwork.freedesktop.org/patch/71852/

When people try igvt-g, one common question we heard is how to get
guest VM's framebuffer. It is for various purposes:

 - A compositor in host (it can be QEMU itself or other viewer
   applications) can use the contents to render a window in host;

 - Remote protocol can easily handle it to support 3D/Media
   accelerated VMs;

The specific requirements include:

 - Be able to map the guest framebuffer so that host CPU can read it;
 - Be able to export guest framebuffer through dam_buf;
 - Be able to direct render with guest framebuffers;

In order to support that, we introduced a new gem object called
gvtbuffer. It is a special object with guest framebuffer's pages as
its backing storage. Meanwhile, it could behave normally like other
gem objects. It can be mapped, exported and used by EGL APIs.

Although we say guest fb pages for gvtbuffer, the solution itself is
safe. Because gvtbuffer gets entries from physical GGTT which cannot
be accessed by guest VM directly. igvt-g device model is responsible
for filling physical GGTT after translating the iova from guest GGTT
table. Even if a malicious guest uses a bad framebuffer, the pages
filled in GGTT are always valid. Then when gvtbuffer tries to get some
entries, they are always valid address not causing hardware problems.

It is possible, however, that the guest VM performs page flip while
gvtbuffer is attached with the framebuffer, and is being used for
rendering. That may cause some tearing in theory. But in practice, we
did not see that. If that is a concern, we can consider to delay the
VBLANK irq injection to guest as a solution.

So in general, do you think it is OK to introduce the gvtbuffer gem
object, or there could be better way to handle it in gem framework?

Currently we have a new IOCTL added for the gvtbuffer, and we also
added some data structures to describe the framebuffer format for user
mode. Do you think that is fine? Thanks again!

Regards,
-Zhiyuan
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Ask for comments of getting guest framebuffer in igvt-g

[Joonas Lahtinen]

Hi,

On to, 2016-03-03 at 17:50 +0800, Zhiyuan Lv wrote:
> Dear i915 developers,
> 
> Here I have one topic hoping to get your comments and suggestions.
> Basically it is about graphics virtualization(igvt-g), for the purpose
> of host system to get virtual machine's framebuffer. We would like to
> hear your opinions about some design opens. Below is the
> patch and some more detailed description. We appreciate your time
> on that, and thanks in advance for any comments!
> 
> https://patchwork.freedesktop.org/patch/71852/
> 
> When people try igvt-g, one common question we heard is how to get
> guest VM's framebuffer. It is for various purposes:
> 
>  - A compositor in host (it can be QEMU itself or other viewer
>    applications) can use the contents to render a window in host;
> 
>  - Remote protocol can easily handle it to support 3D/Media
>    accelerated VMs;
> 
> The specific requirements include:
> 
>  - Be able to map the guest framebuffer so that host CPU can read it;
>  - Be able to export guest framebuffer through dam_buf;
>  - Be able to direct render with guest framebuffers;
> 
> In order to support that, we introduced a new gem object called
> gvtbuffer. It is a special object with guest framebuffer's pages as
> its backing storage. Meanwhile, it could behave normally like other
> gem objects. It can be mapped, exported and used by EGL APIs.
> 
> Although we say guest fb pages for gvtbuffer, the solution itself is
> safe. Because gvtbuffer gets entries from physical GGTT which cannot
> be accessed by guest VM directly. igvt-g device model is responsible
> for filling physical GGTT after translating the iova from guest GGTT
> table. Even if a malicious guest uses a bad framebuffer, the pages
> filled in GGTT are always valid. Then when gvtbuffer tries to get some
> entries, they are always valid address not causing hardware problems.
> 
> It is possible, however, that the guest VM performs page flip while
> gvtbuffer is attached with the framebuffer, and is being used for
> rendering. That may cause some tearing in theory. But in practice, we
> did not see that. If that is a concern, we can consider to delay the
> VBLANK irq injection to guest as a solution.
> 

Could you elborate here? I guess the idea is to make the guest think it
is operating with regular hardware. So I think the owner of a gvtbuffer
will inject the vblank when it has consumed the current framebuffer
data, pretty much like with real hardware. Or what else would trigger
vblank?

Otherwise tearing can not be avoided. Or will the guest be more aware
of being under graphics virtualization than that?

Regards, Joonas

> So in general, do you think it is OK to introduce the gvtbuffer gem
> object, or there could be better way to handle it in gem framework?
> 
> Currently we have a new IOCTL added for the gvtbuffer, and we also
> added some data structures to describe the framebuffer format for user
> mode. Do you think that is fine? Thanks again!
> 
> Regards,
> -Zhiyuan
-- 
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Ask for comments of getting guest framebuffer in igvt-g

[Zhiyuan Lv]

Hi Joonas,

On Fri, Mar 04, 2016 at 04:00:27PM +0200, Joonas Lahtinen wrote:
> Hi,
> 
> On to, 2016-03-03 at 17:50 +0800, Zhiyuan Lv wrote:
> > Dear i915 developers,
> > 
> > Here I have one topic hoping to get your comments and suggestions.
> > Basically it is about graphics virtualization(igvt-g), for the purpose
> > of host system to get virtual machine's framebuffer. We would like to
> > hear your opinions about some design opens. Below is the
> > patch and some more detailed description. We appreciate your time
> > on that, and thanks in advance for any comments!
> > 
> > https://patchwork.freedesktop.org/patch/71852/
> > 
> > When people try igvt-g, one common question we heard is how to get
> > guest VM's framebuffer. It is for various purposes:
> > 
> >  - A compositor in host (it can be QEMU itself or other viewer
> >    applications) can use the contents to render a window in host;
> > 
> >  - Remote protocol can easily handle it to support 3D/Media
> >    accelerated VMs;
> > 
> > The specific requirements include:
> > 
> >  - Be able to map the guest framebuffer so that host CPU can read it;
> >  - Be able to export guest framebuffer through dam_buf;
> >  - Be able to direct render with guest framebuffers;
> > 
> > In order to support that, we introduced a new gem object called
> > gvtbuffer. It is a special object with guest framebuffer's pages as
> > its backing storage. Meanwhile, it could behave normally like other
> > gem objects. It can be mapped, exported and used by EGL APIs.
> > 
> > Although we say guest fb pages for gvtbuffer, the solution itself is
> > safe. Because gvtbuffer gets entries from physical GGTT which cannot
> > be accessed by guest VM directly. igvt-g device model is responsible
> > for filling physical GGTT after translating the iova from guest GGTT
> > table. Even if a malicious guest uses a bad framebuffer, the pages
> > filled in GGTT are always valid. Then when gvtbuffer tries to get some
> > entries, they are always valid address not causing hardware problems.
> > 
> > It is possible, however, that the guest VM performs page flip while
> > gvtbuffer is attached with the framebuffer, and is being used for
> > rendering. That may cause some tearing in theory. But in practice, we
> > did not see that. If that is a concern, we can consider to delay the
> > VBLANK irq injection to guest as a solution.
> > 
> 
> Could you elborate here? I guess the idea is to make the guest think it
> is operating with regular hardware. So I think the owner of a gvtbuffer
> will inject the vblank when it has consumed the current framebuffer
> data, pretty much like with real hardware. Or what else would trigger
> vblank?

The igvt device model is responsible for injecting vblank interrupts to VMs
for i915 running inside. Currently we have a timer for the regular vblank
injection. So if needed, host i915 can notify gvt device model to delay
vblank for a VM, hence delay the flip from VM's point of view. BTW, the idea
of delay is not in current gvt code. We are not sure whether it is a must to
have and whether it is the best way. Just feel that it is doable. Thanks!

Regards,
-Zhiyuan

> 
> Otherwise tearing can not be avoided. Or will the guest be more aware
> of being under graphics virtualization than that?
> 
> Regards, Joonas
> 
> > So in general, do you think it is OK to introduce the gvtbuffer gem
> > object, or there could be better way to handle it in gem framework?
> > 
> > Currently we have a new IOCTL added for the gvtbuffer, and we also
> > added some data structures to describe the framebuffer format for user
> > mode. Do you think that is fine? Thanks again!
> > 
> > Regards,
> > -Zhiyuan
> -- 
> Joonas Lahtinen
> Open Source Technology Center
> Intel Corporation
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Ask for comments of getting guest framebuffer in igvt-g

[Joonas Lahtinen]

Hi,

On pe, 2016-03-04 at 23:38 +0800, Zhiyuan Lv wrote:
> Hi Joonas,
> 
> On Fri, Mar 04, 2016 at 04:00:27PM +0200, Joonas Lahtinen wrote:
> > 
> > Hi,
> > 
> > On to, 2016-03-03 at 17:50 +0800, Zhiyuan Lv wrote:
> > > 
> > > Dear i915 developers,
> > > 
> > > Here I have one topic hoping to get your comments and suggestions.
> > > Basically it is about graphics virtualization(igvt-g), for the purpose
> > > of host system to get virtual machine's framebuffer. We would like to
> > > hear your opinions about some design opens. Below is the
> > > patch and some more detailed description. We appreciate your time
> > > on that, and thanks in advance for any comments!
> > > 
> > > https://patchwork.freedesktop.org/patch/71852/
> > > 
> > > When people try igvt-g, one common question we heard is how to get
> > > guest VM's framebuffer. It is for various purposes:
> > > 
> > >  - A compositor in host (it can be QEMU itself or other viewer
> > >    applications) can use the contents to render a window in host;
> > > 
> > >  - Remote protocol can easily handle it to support 3D/Media
> > >    accelerated VMs;
> > > 
> > > The specific requirements include:
> > > 
> > >  - Be able to map the guest framebuffer so that host CPU can read it;
> > >  - Be able to export guest framebuffer through dam_buf;
> > >  - Be able to direct render with guest framebuffers;
> > > 
> > > In order to support that, we introduced a new gem object called
> > > gvtbuffer. It is a special object with guest framebuffer's pages as
> > > its backing storage. Meanwhile, it could behave normally like other
> > > gem objects. It can be mapped, exported and used by EGL APIs.
> > > 
> > > Although we say guest fb pages for gvtbuffer, the solution itself is
> > > safe. Because gvtbuffer gets entries from physical GGTT which cannot
> > > be accessed by guest VM directly. igvt-g device model is responsible
> > > for filling physical GGTT after translating the iova from guest GGTT
> > > table. Even if a malicious guest uses a bad framebuffer, the pages
> > > filled in GGTT are always valid. Then when gvtbuffer tries to get some
> > > entries, they are always valid address not causing hardware problems.
> > > 
> > > It is possible, however, that the guest VM performs page flip while
> > > gvtbuffer is attached with the framebuffer, and is being used for
> > > rendering. That may cause some tearing in theory. But in practice, we
> > > did not see that. If that is a concern, we can consider to delay the
> > > VBLANK irq injection to guest as a solution.
> > > 
> > Could you elborate here? I guess the idea is to make the guest think it
> > is operating with regular hardware. So I think the owner of a gvtbuffer
> > will inject the vblank when it has consumed the current framebuffer
> > data, pretty much like with real hardware. Or what else would trigger
> > vblank?
> The igvt device model is responsible for injecting vblank interrupts to VMs
> for i915 running inside. Currently we have a timer for the regular vblank
> injection. So if needed, host i915 can notify gvt device model to delay
> vblank for a VM, hence delay the flip from VM's point of view. BTW, the idea
> of delay is not in current gvt code. We are not sure whether it is a must to
> have and whether it is the best way. Just feel that it is doable. Thanks!
> 

The timer best follows the "real hardware" imitation scenario, so it
should be good. Would it be an option to triple-buffer, keeping a copy
of the frame intact which the DOM0 is drawing? Or is that too harsh on
memory requirements.

That would avoid tearing, but also let the guest freely update at
constant rate. From DOM0, the compositor or whatever application
consumes the frames, would just observe frame skipping instead of
tearing.

Regards, Joonas

> Regards,
> -Zhiyuan
> 
> > 
> > 
> > Otherwise tearing can not be avoided. Or will the guest be more aware
> > of being under graphics virtualization than that?
> > 
> > Regards, Joonas
> > 
> > > 
> > > So in general, do you think it is OK to introduce the gvtbuffer gem
> > > object, or there could be better way to handle it in gem framework?
> > > 
> > > Currently we have a new IOCTL added for the gvtbuffer, and we also
> > > added some data structures to describe the framebuffer format for user
> > > mode. Do you think that is fine? Thanks again!
> > > 
> > > Regards,
> > > -Zhiyuan
-- 
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Ask for comments of getting guest framebuffer in igvt-g

[Zhiyuan Lv]

Hi Joonas,

On Mon, Mar 07, 2016 at 12:20:48PM +0200, Joonas Lahtinen wrote:
> Hi,
> 
> On pe, 2016-03-04 at 23:38 +0800, Zhiyuan Lv wrote:
> > Hi Joonas,
> > 
> > On Fri, Mar 04, 2016 at 04:00:27PM +0200, Joonas Lahtinen wrote:
> > > 
> > > Hi,
> > > 
> > > On to, 2016-03-03 at 17:50 +0800, Zhiyuan Lv wrote:
> > > > 
> > > > Dear i915 developers,
> > > > 
> > > > Here I have one topic hoping to get your comments and suggestions.
> > > > Basically it is about graphics virtualization(igvt-g), for the purpose
> > > > of host system to get virtual machine's framebuffer. We would like to
> > > > hear your opinions about some design opens. Below is the
> > > > patch and some more detailed description. We appreciate your time
> > > > on that, and thanks in advance for any comments!
> > > > 
> > > > https://patchwork.freedesktop.org/patch/71852/
> > > > 
> > > > When people try igvt-g, one common question we heard is how to get
> > > > guest VM's framebuffer. It is for various purposes:
> > > > 
> > > >  - A compositor in host (it can be QEMU itself or other viewer
> > > >    applications) can use the contents to render a window in host;
> > > > 
> > > >  - Remote protocol can easily handle it to support 3D/Media
> > > >    accelerated VMs;
> > > > 
> > > > The specific requirements include:
> > > > 
> > > >  - Be able to map the guest framebuffer so that host CPU can read it;
> > > >  - Be able to export guest framebuffer through dam_buf;
> > > >  - Be able to direct render with guest framebuffers;
> > > > 
> > > > In order to support that, we introduced a new gem object called
> > > > gvtbuffer. It is a special object with guest framebuffer's pages as
> > > > its backing storage. Meanwhile, it could behave normally like other
> > > > gem objects. It can be mapped, exported and used by EGL APIs.
> > > > 
> > > > Although we say guest fb pages for gvtbuffer, the solution itself is
> > > > safe. Because gvtbuffer gets entries from physical GGTT which cannot
> > > > be accessed by guest VM directly. igvt-g device model is responsible
> > > > for filling physical GGTT after translating the iova from guest GGTT
> > > > table. Even if a malicious guest uses a bad framebuffer, the pages
> > > > filled in GGTT are always valid. Then when gvtbuffer tries to get some
> > > > entries, they are always valid address not causing hardware problems.
> > > > 
> > > > It is possible, however, that the guest VM performs page flip while
> > > > gvtbuffer is attached with the framebuffer, and is being used for
> > > > rendering. That may cause some tearing in theory. But in practice, we
> > > > did not see that. If that is a concern, we can consider to delay the
> > > > VBLANK irq injection to guest as a solution.
> > > > 
> > > Could you elborate here? I guess the idea is to make the guest think it
> > > is operating with regular hardware. So I think the owner of a gvtbuffer
> > > will inject the vblank when it has consumed the current framebuffer
> > > data, pretty much like with real hardware. Or what else would trigger
> > > vblank?
> > The igvt device model is responsible for injecting vblank interrupts to VMs
> > for i915 running inside. Currently we have a timer for the regular vblank
> > injection. So if needed, host i915 can notify gvt device model to delay
> > vblank for a VM, hence delay the flip from VM's point of view. BTW, the idea
> > of delay is not in current gvt code. We are not sure whether it is a must to
> > have and whether it is the best way. Just feel that it is doable. Thanks!
> > 
> 
> The timer best follows the "real hardware" imitation scenario, so it
> should be good. Would it be an option to triple-buffer, keeping a copy
> of the frame intact which the DOM0 is drawing? Or is that too harsh on
> memory requirements.
> 
> That would avoid tearing, but also let the guest freely update at
> constant rate. From DOM0, the compositor or whatever application
> consumes the frames, would just observe frame skipping instead of
> tearing.

Thanks for the idea! Keeping a copy could avoid the tearing
completely. Following that approach, we may not need the special
gvtbuffer gem object. We can introduce an IOCTL to copy a VM's
framebuffer to a given normal gem object, then make sure that during
the copy, guest driver in VM will not modify the fb.

The only concern here is the performance impact, since there have to
be memory copy for each frame. I do not have data in hand, but guess
that might be quite significant. Thanks!

Regards,
-Zhiyuan

> 
> Regards, Joonas
> 
> > Regards,
> > -Zhiyuan
> > 
> > > 
> > > 
> > > Otherwise tearing can not be avoided. Or will the guest be more aware
> > > of being under graphics virtualization than that?
> > > 
> > > Regards, Joonas
> > > 
> > > > 
> > > > So in general, do you think it is OK to introduce the gvtbuffer gem
> > > > object, or there could be better way to handle it in gem framework?
> > > > 
> > > > Currently we have a new IOCTL added for the gvtbuffer, and we also
> > > > added some data structures to describe the framebuffer format for user
> > > > mode. Do you think that is fine? Thanks again!
> > > > 
> > > > Regards,
> > > > -Zhiyuan
> -- 
> Joonas Lahtinen
> Open Source Technology Center
> Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [iGVT-g] Ask for comments of getting guest framebuffer in igvt-g

[Tian, Kevin]

> From: Zhiyuan Lv
> Sent: Tuesday, March 08, 2016 9:26 AM
> 
> > > The igvt device model is responsible for injecting vblank interrupts to VMs
> > > for i915 running inside. Currently we have a timer for the regular vblank
> > > injection. So if needed, host i915 can notify gvt device model to delay
> > > vblank for a VM, hence delay the flip from VM's point of view. BTW, the idea
> > > of delay is not in current gvt code. We are not sure whether it is a must to
> > > have and whether it is the best way. Just feel that it is doable. Thanks!
> > >
> >
> > The timer best follows the "real hardware" imitation scenario, so it
> > should be good. Would it be an option to triple-buffer, keeping a copy
> > of the frame intact which the DOM0 is drawing? Or is that too harsh on
> > memory requirements.
> >
> > That would avoid tearing, but also let the guest freely update at
> > constant rate. From DOM0, the compositor or whatever application
> > consumes the frames, would just observe frame skipping instead of
> > tearing.
> 
> Thanks for the idea! Keeping a copy could avoid the tearing
> completely. Following that approach, we may not need the special
> gvtbuffer gem object. We can introduce an IOCTL to copy a VM's
> framebuffer to a given normal gem object, then make sure that during
> the copy, guest driver in VM will not modify the fb.
> 
> The only concern here is the performance impact, since there have to
> be memory copy for each frame. I do not have data in hand, but guess
> that might be quite significant. Thanks!
> 

memory copy is very slow (especially when aperture is mapped as WC
which means uncatchable). If necessary we should use GPU to blit the
content however it then means you still need a gem object. :-)

btw I don't think this vblank issue would be very significant. The main
targeted usage of GVT-g is for server virtualization/cloud, where 
a remoting protocol is required for customer to see the content through
network. My feeling is that tearing should be well considered already by
those remoting protocols, regardless of how vblank is virtualized.

Thanks
Kevin
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [iGVT-g] Ask for comments of getting guest framebuffer in igvt-g

[Gerd Hoffmann]

  Hi,

> btw I don't think this vblank issue would be very significant. The main
> targeted usage of GVT-g is for server virtualization/cloud, where 
> a remoting protocol is required for customer to see the content through
> network.

The plan for that is to let the gpu video encoder process the guest
framebuffer.  So the video encoder still being busy processing the
buffer while the guest already started updating it can certainly lead to
visible tearing.

I think using vblank to signal the guest when the host is done
processing the framebuffer makes sense.  It'll simply throttle the guest
in case the host is too busy to keep up.

Storing the guest framebuffer via blit somewhere else, just to allow the
guest render at full frame rate even if the host isn't able to process
the frames fast enough looks pointless to me.  We are simply wasting gpu
cycles then.

cheers,
  Gerd

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [iGVT-g] Ask for comments of getting guest framebuffer in igvt-g

[Tian, Kevin]

> From: Gerd Hoffmann [mailto:kraxel@redhat.com]
> Sent: Tuesday, March 08, 2016 4:37 PM
> 
>   Hi,
> 
> > btw I don't think this vblank issue would be very significant. The main
> > targeted usage of GVT-g is for server virtualization/cloud, where
> > a remoting protocol is required for customer to see the content through
> > network.
> 
> The plan for that is to let the gpu video encoder process the guest
> framebuffer.  So the video encoder still being busy processing the
> buffer while the guest already started updating it can certainly lead to
> visible tearing.
> 
> I think using vblank to signal the guest when the host is done
> processing the framebuffer makes sense.  It'll simply throttle the guest
> in case the host is too busy to keep up.

Thanks Gerd for elaborating this part. We can have this requirement
considered once this new interface is agreed here. There may need
more consideration how to align "host is done processing" with virtual
vblank delivery. 

Thanks
Kevin
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [iGVT-g] Ask for comments of getting guest framebuffer in igvt-g

[Joonas Lahtinen]

On ti, 2016-03-08 at 09:36 +0100, Gerd Hoffmann wrote:
>   Hi,
> 
> > 
> > btw I don't think this vblank issue would be very significant. The main
> > targeted usage of GVT-g is for server virtualization/cloud, where 
> > a remoting protocol is required for customer to see the content through
> > network.

There is also the use scenario of locally scanning out the buffers with
real hardware, the DOM0 working as a kind of KVM switch.

> The plan for that is to let the gpu video encoder process the guest
> framebuffer.  So the video encoder still being busy processing the
> buffer while the guest already started updating it can certainly lead to
> visible tearing.
> 
> I think using vblank to signal the guest when the host is done
> processing the framebuffer makes sense.  It'll simply throttle the guest
> in case the host is too busy to keep up.
> 

This was my initial suggestion, but I could see this exposing plenty of
behavior on the guest system that is not covered by regular testing.

> Storing the guest framebuffer via blit somewhere else, just to allow the
> guest render at full frame rate even if the host isn't able to process
> the frames fast enough looks pointless to me.  We are simply wasting gpu
> cycles then.

This just happens to be the most non-invasive way of doing it, where
guest will behave much like on real hardware.

I'm not taking a stance for triple buffering, but I think it's
important that we do not consider just a single use-case and rule other
options out. Especially if the timer system is already been tested some
 and could serve both scenarios.

Regards, Joonas

> 
> cheers,
>   Gerd
> 
-- 
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Ask for comments of getting guest framebuffer in igvt-g

[Tian, Kevin]

> From: Zhiyuan Lv
> Sent: Thursday, March 03, 2016 5:51 PM
> 
> Dear i915 developers,
> 
> Here I have one topic hoping to get your comments and suggestions.
> Basically it is about graphics virtualization(igvt-g), for the purpose
> of host system to get virtual machine's framebuffer. We would like to
> hear your opinions about some design opens. Below is the
> patch and some more detailed description. We appreciate your time
> on that, and thanks in advance for any comments!
> 
> https://patchwork.freedesktop.org/patch/71852/
> 
> When people try igvt-g, one common question we heard is how to get
> guest VM's framebuffer. It is for various purposes:
> 
>  - A compositor in host (it can be QEMU itself or other viewer
>    applications) can use the contents to render a window in host;
> 
>  - Remote protocol can easily handle it to support 3D/Media
>    accelerated VMs;
> 
> The specific requirements include:
> 
>  - Be able to map the guest framebuffer so that host CPU can read it;
>  - Be able to export guest framebuffer through dam_buf;
>  - Be able to direct render with guest framebuffers;
> 
> In order to support that, we introduced a new gem object called
> gvtbuffer. It is a special object with guest framebuffer's pages as
> its backing storage. Meanwhile, it could behave normally like other
> gem objects. It can be mapped, exported and used by EGL APIs.
> 
> Although we say guest fb pages for gvtbuffer, the solution itself is
> safe. Because gvtbuffer gets entries from physical GGTT which cannot
> be accessed by guest VM directly. igvt-g device model is responsible
> for filling physical GGTT after translating the iova from guest GGTT
> table. Even if a malicious guest uses a bad framebuffer, the pages
> filled in GGTT are always valid. Then when gvtbuffer tries to get some
> entries, they are always valid address not causing hardware problems.
> 
> It is possible, however, that the guest VM performs page flip while
> gvtbuffer is attached with the framebuffer, and is being used for
> rendering. That may cause some tearing in theory. But in practice, we
> did not see that. If that is a concern, we can consider to delay the
> VBLANK irq injection to guest as a solution.
> 
> So in general, do you think it is OK to introduce the gvtbuffer gem
> object, or there could be better way to handle it in gem framework?
> 
> Currently we have a new IOCTL added for the gvtbuffer, and we also
> added some data structures to describe the framebuffer format for user
> mode. Do you think that is fine? Thanks again!
> 

Hi, Zhiyuan,

After reading the patchwork link, is my below understanding correct
regarding to the key logic of this new IOCTL?

- It's similar to stolen memory, i.e. the backing storage may not be
directly accessed by the driver (it's other VM's memory) so no
'page struct' is available;

- The sg_dma_address of the backing storage is retrieved directly
from GGTT entries corresponding to the gmadr of guest framebuffer,
(those entries are audited by GVT-g device model before programming 
GGTT);

- Then the gem object can be pinned to either GGTT or PPGTT, upon
request from user-level compositor;

- A notification will be sent by GVT-g device model, upon any change
of the guest framebuffer location,  (including change of underlying GGTT 
entry), but this notification is not implemented yet in this RFC patch;

- Upon such notification, user-level compositor is expected to destroy
previous gem object and then recreate a new object according to the
new information;

One additional comment here. Since this gem object implies another
reference to the guest memory page, we need a step to call into
GVT-g device model to claim such reference (which will then lead to a
refcnt increment of the guest page through hypervisor specific way). 
Today it's optional since our device model claims reference of all
guest memory pages in a batch at boot time, which however might
be optimized in the future to do selective claim so within device model
we need clearly mark out all explicit references. 

Thanks
Kevin
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Ask for comments of getting guest framebuffer in igvt-g

[Zhiyuan Lv]

Hi Kevin,

On Tue, Mar 08, 2016 at 10:44:39AM +0800, Tian, Kevin wrote:
> > From: Zhiyuan Lv
> > Sent: Thursday, March 03, 2016 5:51 PM
> > 
> > Dear i915 developers,
> > 
> > Here I have one topic hoping to get your comments and suggestions.
> > Basically it is about graphics virtualization(igvt-g), for the purpose
> > of host system to get virtual machine's framebuffer. We would like to
> > hear your opinions about some design opens. Below is the
> > patch and some more detailed description. We appreciate your time
> > on that, and thanks in advance for any comments!
> > 
> > https://patchwork.freedesktop.org/patch/71852/
> > 
> > When people try igvt-g, one common question we heard is how to get
> > guest VM's framebuffer. It is for various purposes:
> > 
> >  - A compositor in host (it can be QEMU itself or other viewer
> >    applications) can use the contents to render a window in host;
> > 
> >  - Remote protocol can easily handle it to support 3D/Media
> >    accelerated VMs;
> > 
> > The specific requirements include:
> > 
> >  - Be able to map the guest framebuffer so that host CPU can read it;
> >  - Be able to export guest framebuffer through dam_buf;
> >  - Be able to direct render with guest framebuffers;
> > 
> > In order to support that, we introduced a new gem object called
> > gvtbuffer. It is a special object with guest framebuffer's pages as
> > its backing storage. Meanwhile, it could behave normally like other
> > gem objects. It can be mapped, exported and used by EGL APIs.
> > 
> > Although we say guest fb pages for gvtbuffer, the solution itself is
> > safe. Because gvtbuffer gets entries from physical GGTT which cannot
> > be accessed by guest VM directly. igvt-g device model is responsible
> > for filling physical GGTT after translating the iova from guest GGTT
> > table. Even if a malicious guest uses a bad framebuffer, the pages
> > filled in GGTT are always valid. Then when gvtbuffer tries to get some
> > entries, they are always valid address not causing hardware problems.
> > 
> > It is possible, however, that the guest VM performs page flip while
> > gvtbuffer is attached with the framebuffer, and is being used for
> > rendering. That may cause some tearing in theory. But in practice, we
> > did not see that. If that is a concern, we can consider to delay the
> > VBLANK irq injection to guest as a solution.
> > 
> > So in general, do you think it is OK to introduce the gvtbuffer gem
> > object, or there could be better way to handle it in gem framework?
> > 
> > Currently we have a new IOCTL added for the gvtbuffer, and we also
> > added some data structures to describe the framebuffer format for user
> > mode. Do you think that is fine? Thanks again!
> > 
> 
> Hi, Zhiyuan,
> 
> After reading the patchwork link, is my below understanding correct
> regarding to the key logic of this new IOCTL?
> 
> - It's similar to stolen memory, i.e. the backing storage may not be
> directly accessed by the driver (it's other VM's memory) so no
> 'page struct' is available;
> 
> - The sg_dma_address of the backing storage is retrieved directly
> from GGTT entries corresponding to the gmadr of guest framebuffer,
> (those entries are audited by GVT-g device model before programming 
> GGTT);
> 
> - Then the gem object can be pinned to either GGTT or PPGTT, upon
> request from user-level compositor;

Above are all correct.

> 
> - A notification will be sent by GVT-g device model, upon any change
> of the guest framebuffer location,  (including change of underlying GGTT 
> entry), but this notification is not implemented yet in this RFC patch;
> 
> - Upon such notification, user-level compositor is expected to destroy
> previous gem object and then recreate a new object according to the
> new information;

That's right. I did not mention this part to make current discussion specific
:-)

> 
> One additional comment here. Since this gem object implies another
> reference to the guest memory page, we need a step to call into
> GVT-g device model to claim such reference (which will then lead to a
> refcnt increment of the guest page through hypervisor specific way). 
> Today it's optional since our device model claims reference of all
> guest memory pages in a batch at boot time, which however might
> be optimized in the future to do selective claim so within device model
> we need clearly mark out all explicit references.

Noted. Thanks for the comments!

Regards,
-Zhiyuan

> 
> Thanks
> Kevin
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx