Sharing display between guests

Sharing display between guests

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 2154 bytes --]

Hi,

I've started using Xen on an Allwinner A33, which works great as an
headless device using the latest PSCI patches in U-Boot.

However, we would like to do something more with it, and we would need
to have two VMs accessing the display at once, each one drawing in its
own part of the framebuffer.

Something that would look like this:

   Framebuffer
+---------------+
|               |
|    Guest 1    |
|               |
+---------------+
|               |
|               |
|               |
|    Guest 2    |
|               |
|               |
|               |
+---------------+

Where thing start to get interesting is that the second guest would be
running Android, and as such would need OpenGL support, and access to
the GPU, and that ideally the first guest would need to be able to
draw over all the screen to create some kind of a drop-down menu.

Our first thought was to use two different planes of a DRM/KMS driver,
one for each VM, with the second guest having the primary plane, and
the first guest having an overlay, and we would set it up in dom0.

That would mean that we would have a static "composition", that would
be setup once and we could forget about it during the life of the
system.

This way, we would also have a fixed size framebuffer assigned to
Android, which is much easier to support, and since we have total
control over the application in the first guest, we would be able to
control how much "transparency" we want to leave (== how much of
Android do we want to be displayed), and we would be able to create
our drop-down menu.

Now the hard part: is such a setup possible at all with Xen? Can we
export a single plane to a guest and let it be the only user of it? If
that is possible, how would that interact with the 3D acceleration?

If not, is it something that is conceptually flawed, or does one just
need to write the appropriate amount of code? Do you have a better
solution to this problem?

Thanks a lot for your feedback,
Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1177 bytes --]

On Thu, 2015-07-02 at 13:13 +0200, Maxime Ripard wrote:
> Hi,
> 
Hi,

I don't have much comments on all this, just that, reading, it, it
sounded somewhat similar to what GL is doing:

http://www.xenproject.org/component/allvideoshare/video/latest/ces15-globalogic.html
http://www.xenproject.org/component/allvideoshare/video/globallogic-xen-android.html

> That would mean that we would have a static "composition", that would
> be setup once and we could forget about it during the life of the
> system.
> 
Well, they use a DomU and Dom0, rather than 2 DomUs (or so I think), but
is seems to me that at least some composition is indeed happening.

I don't have much more details, I'm afraid, but if you think your use
case is related, or at least that you could benefit from interacting
with them and sharing experiences/code/whatever, just ask, I've Cc-ed
some of them. :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 1514 bytes --]

Hi Dario

On Thu, Jul 02, 2015 at 02:18:18PM +0200, Dario Faggioli wrote:
> I don't have much comments on all this, just that, reading, it, it
> sounded somewhat similar to what GL is doing:
> 
> http://www.xenproject.org/component/allvideoshare/video/latest/ces15-globalogic.html
> http://www.xenproject.org/component/allvideoshare/video/globallogic-xen-android.html

For some reason, the first video wouldn't play here, so I can really
comment on the second one, but I don't think there was a screen
sharing here. It demoes Android running in a domU, which is already a
nice achievement, but it's not really the part that concern me the
most. I'm slightly more concerned about the two VM displaying stuff at
the same time.

> > That would mean that we would have a static "composition", that would
> > be setup once and we could forget about it during the life of the
> > system.
>
> Well, they use a DomU and Dom0, rather than 2 DomUs (or so I think), but
> is seems to me that at least some composition is indeed happening.

Ideally, we would like both to be domU, with dom0 only taking care of
the composition.

> I don't have much more details, I'm afraid, but if you think your use
> case is related, or at least that you could benefit from interacting
> with them and sharing experiences/code/whatever, just ask, I've Cc-ed
> some of them. :-)

Thanks!
Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Ian Campbell]

(switching to my work address) 

On Thu, 2015-07-02 at 13:13 +0200, Maxime Ripard wrote:
> Hi,
> 
> I've started using Xen on an Allwinner A33, which works great as an
> headless device using the latest PSCI patches in U-Boot.
> 
> However, we would like to do something more with it, and we would need
> to have two VMs accessing the display at once, each one drawing in its
> own part of the framebuffer.
> 
> Something that would look like this:
> 
>    Framebuffer
> +---------------+
> |               |
> |    Guest 1    |
> |               |
> +---------------+
> |               |
> |               |
> |               |
> |    Guest 2    |
> |               |
> |               |
> |               |
> +---------------+
> 
> Where thing start to get interesting is that the second guest would be
> running Android, and as such would need OpenGL support, and access to
> the GPU, and that ideally the first guest would need to be able to
> draw over all the screen to create some kind of a drop-down menu.
> 
> Our first thought was to use two different planes of a DRM/KMS driver,
> one for each VM, with the second guest having the primary plane, and
> the first guest having an overlay, and we would set it up in dom0.
> 
> That would mean that we would have a static "composition", that would
> be setup once and we could forget about it during the life of the
> system.
> 
> This way, we would also have a fixed size framebuffer assigned to
> Android, which is much easier to support, and since we have total
> control over the application in the first guest, we would be able to
> control how much "transparency" we want to leave (== how much of
> Android do we want to be displayed), and we would be able to create
> our drop-down menu.
> 
> Now the hard part: is such a setup possible at all with Xen? Can we
> export a single plane to a guest and let it be the only user of it? If
> that is possible, how would that interact with the 3D acceleration?

As it stands today Xen supports exposing a 2D only PV graphics device
(called "pvfb", essentially a flat/linear framebuffer) to guests, I
think this has limited 2D acceleration support let alone 3D support. You
can also (at least in principal, i.e. if the hardware isn't too strange)
pass a GPU through to exactly one guest (often it's just dom0).

Some people have also (I think) managed to use pvfb to expose a real
linear fb to a guest, e.g. a "surface" in the real GPU which is setup by
some more privileged domain.

But, it sounds like you really want a way to expose the GPU
functionality (i.e. 3D support) to at least one if not both guests.

Are you expecting Guest 1 and Guest 2 to be isolated from each other in
a secure way? Or is one considered to be more privileged than the other?

(disclaimer: From here on I'm a bit outside the realms of things I'm
getting even further from the space where I'm confident in my own
understanding...)

Various people have tried to solve this usecase by providing a PV 3D
(OpenGL, Gallium, etc) protocol over the PV FB, but I think this turns
out to be quite tricky (in terms of shuffling textures around as well as
being able to validate the command stream) and generally little comes of
it beyond a PoC. (It's been quite some time since I've seen someone try
this approach).

The other approach would involve exposing the actual hardware to the
guest(s). In general unless your hardware has some special features
(vGPU or SR-IOV or something similar) it is rarely safe to let two
guests access the same bit of h/w unless they are both willing to
co-operate and are assumed to be mutually destructive.

Various folks, including Globalogic who Dario pointed too and Intel with
XenGT (which has been renamed but I don't recall the new name) have
tried to take a hybrid approach emulating parts of the device in some
privileged layer and exposing some parts of the h/w directly to the
guest. AIUI this is necessarily specific to a particular GPU device and
needs the device to have some convenient properties, but falls short of
requiring full "virtualisability" of the h/w ala vGPU or SR-IOV.

> If not, is it something that is conceptually flawed, or does one just
> need to write the appropriate amount of code? Do you have a better
> solution to this problem?

I don't think it is conceptually flawed, but I also think it's not an
easy problem (sorry).

> 
> Thanks a lot for your feedback,
> Maxime
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 5694 bytes --]

Hi Ian,

On Mon, Jul 06, 2015 at 12:53:49PM +0100, Ian Campbell wrote:
> (switching to my work address) 
> 
> On Thu, 2015-07-02 at 13:13 +0200, Maxime Ripard wrote:
> > Hi,
> > 
> > I've started using Xen on an Allwinner A33, which works great as an
> > headless device using the latest PSCI patches in U-Boot.
> > 
> > However, we would like to do something more with it, and we would need
> > to have two VMs accessing the display at once, each one drawing in its
> > own part of the framebuffer.
> > 
> > Something that would look like this:
> > 
> >    Framebuffer
> > +---------------+
> > |               |
> > |    Guest 1    |
> > |               |
> > +---------------+
> > |               |
> > |               |
> > |               |
> > |    Guest 2    |
> > |               |
> > |               |
> > |               |
> > +---------------+
> > 
> > Where thing start to get interesting is that the second guest would be
> > running Android, and as such would need OpenGL support, and access to
> > the GPU, and that ideally the first guest would need to be able to
> > draw over all the screen to create some kind of a drop-down menu.
> > 
> > Our first thought was to use two different planes of a DRM/KMS driver,
> > one for each VM, with the second guest having the primary plane, and
> > the first guest having an overlay, and we would set it up in dom0.
> > 
> > That would mean that we would have a static "composition", that would
> > be setup once and we could forget about it during the life of the
> > system.
> > 
> > This way, we would also have a fixed size framebuffer assigned to
> > Android, which is much easier to support, and since we have total
> > control over the application in the first guest, we would be able to
> > control how much "transparency" we want to leave (== how much of
> > Android do we want to be displayed), and we would be able to create
> > our drop-down menu.
> > 
> > Now the hard part: is such a setup possible at all with Xen? Can we
> > export a single plane to a guest and let it be the only user of it? If
> > that is possible, how would that interact with the 3D acceleration?
> 
> As it stands today Xen supports exposing a 2D only PV graphics device
> (called "pvfb", essentially a flat/linear framebuffer) to guests, I
> think this has limited 2D acceleration support let alone 3D support. You
> can also (at least in principal, i.e. if the hardware isn't too strange)
> pass a GPU through to exactly one guest (often it's just dom0).
> 
> Some people have also (I think) managed to use pvfb to expose a real
> linear fb to a guest, e.g. a "surface" in the real GPU which is setup by
> some more privileged domain.
> 
> But, it sounds like you really want a way to expose the GPU
> functionality (i.e. 3D support) to at least one if not both guests.
> 
> Are you expecting Guest 1 and Guest 2 to be isolated from each other in
> a secure way? Or is one considered to be more privileged than the other?

I'd like to have both guests as isolated from each other as possible.

But there's two sub-issues here actually:

1) Being able to have two VMs showing stuff on the same framebuffer at
the same time, without being aware of the other. If we could make pvfb
run on a DRM/KMS plane, that would be perfect for us. Another solution
would be to split the framebuffer in half and have pvfb running on
those two halves.

2) Then, one of the guests would be Android, which really requires
OpenGL, but the second guests would ideally use it as well, but if it
complicates things (and it really seems like it does), we can drop
that from the requirement, which would make us fall in the "exactly
one guest" case you were talking about (except that it wouldn't dom0,
but one of the domU's that would access it).

Our hardware is an A33-based board, that uses a Mali. I'm not exactly
sure about what you meant with your "if the hardware isn't too
strange". Is there features or hints I should be looking for?

> (disclaimer: From here on I'm a bit outside the realms of things I'm
> getting even further from the space where I'm confident in my own
> understanding...)
> 
> Various people have tried to solve this usecase by providing a PV 3D
> (OpenGL, Gallium, etc) protocol over the PV FB, but I think this turns
> out to be quite tricky (in terms of shuffling textures around as well as
> being able to validate the command stream) and generally little comes of
> it beyond a PoC. (It's been quite some time since I've seen someone try
> this approach).
> 
> The other approach would involve exposing the actual hardware to the
> guest(s). In general unless your hardware has some special features
> (vGPU or SR-IOV or something similar) it is rarely safe to let two
> guests access the same bit of h/w unless they are both willing to
> co-operate and are assumed to be mutually destructive.
> 
> Various folks, including Globalogic who Dario pointed too and Intel with
> XenGT (which has been renamed but I don't recall the new name) have
> tried to take a hybrid approach emulating parts of the device in some
> privileged layer and exposing some parts of the h/w directly to the
> guest. AIUI this is necessarily specific to a particular GPU device and
> needs the device to have some convenient properties, but falls short of
> requiring full "virtualisability" of the h/w ala vGPU or SR-IOV.

Ok, so I guess that rules out entirely being able to use the GPUs from
both VMs.

Thanks!
Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Ian Campbell]

On Mon, 2015-07-06 at 14:26 +0200, Maxime Ripard wrote:
> Hi Ian,
> 
> On Mon, Jul 06, 2015 at 12:53:49PM +0100, Ian Campbell wrote:
> > (switching to my work address) 
> > 
> > On Thu, 2015-07-02 at 13:13 +0200, Maxime Ripard wrote:
> > > Hi,
> > > 
> > > I've started using Xen on an Allwinner A33, which works great as an
> > > headless device using the latest PSCI patches in U-Boot.
> > > 
> > > However, we would like to do something more with it, and we would need
> > > to have two VMs accessing the display at once, each one drawing in its
> > > own part of the framebuffer.
> > > 
> > > Something that would look like this:
> > > 
> > >    Framebuffer
> > > +---------------+
> > > |               |
> > > |    Guest 1    |
> > > |               |
> > > +---------------+
> > > |               |
> > > |               |
> > > |               |
> > > |    Guest 2    |
> > > |               |
> > > |               |
> > > |               |
> > > +---------------+
> > > 
> > > Where thing start to get interesting is that the second guest would be
> > > running Android, and as such would need OpenGL support, and access to
> > > the GPU, and that ideally the first guest would need to be able to
> > > draw over all the screen to create some kind of a drop-down menu.
> > > 
> > > Our first thought was to use two different planes of a DRM/KMS driver,
> > > one for each VM, with the second guest having the primary plane, and
> > > the first guest having an overlay, and we would set it up in dom0.
> > > 
> > > That would mean that we would have a static "composition", that would
> > > be setup once and we could forget about it during the life of the
> > > system.
> > > 
> > > This way, we would also have a fixed size framebuffer assigned to
> > > Android, which is much easier to support, and since we have total
> > > control over the application in the first guest, we would be able to
> > > control how much "transparency" we want to leave (== how much of
> > > Android do we want to be displayed), and we would be able to create
> > > our drop-down menu.
> > > 
> > > Now the hard part: is such a setup possible at all with Xen? Can we
> > > export a single plane to a guest and let it be the only user of it? If
> > > that is possible, how would that interact with the 3D acceleration?
> > 
> > As it stands today Xen supports exposing a 2D only PV graphics device
> > (called "pvfb", essentially a flat/linear framebuffer) to guests, I
> > think this has limited 2D acceleration support let alone 3D support. You
> > can also (at least in principal, i.e. if the hardware isn't too strange)
> > pass a GPU through to exactly one guest (often it's just dom0).
> > 
> > Some people have also (I think) managed to use pvfb to expose a real
> > linear fb to a guest, e.g. a "surface" in the real GPU which is setup by
> > some more privileged domain.
> > 
> > But, it sounds like you really want a way to expose the GPU
> > functionality (i.e. 3D support) to at least one if not both guests.
> > 
> > Are you expecting Guest 1 and Guest 2 to be isolated from each other in
> > a secure way? Or is one considered to be more privileged than the other?
> 
> I'd like to have both guests as isolated from each other as possible.
> 
> But there's two sub-issues here actually:
> 
> 1) Being able to have two VMs showing stuff on the same framebuffer at
> the same time, without being aware of the other. If we could make pvfb
> run on a DRM/KMS plane, that would be perfect for us. Another solution
> would be to split the framebuffer in half and have pvfb running on
> those two halves.

Right, this portion is a bit more tractable in the short term I think.

> 2) Then, one of the guests would be Android, which really requires
> OpenGL, but the second guests would ideally use it as well, but if it
> complicates things (and it really seems like it does), we can drop
> that from the requirement, which would make us fall in the "exactly
> one guest" case you were talking about (except that it wouldn't dom0,
> but one of the domU's that would access it).

IOW give the Android full access to the h/w and have it provide a 2d
pvfb to the other guest, yes I think that could be made to work.

However, there is a wrinkle, which is that passthrough normally requires
an SMMU, which I'm reasonably sure the A33 doesn't have. The usual
workaround (which I think Glaballogic use) is to allocate guest memory
1:1 (i.e. so it matches in IPA and PA space). This is a hack though and
we don't like it :-/.

> Our hardware is an A33-based board, that uses a Mali. I'm not exactly
> sure about what you meant with your "if the hardware isn't too
> strange". Is there features or hints I should be looking for?

That was mostly a reference to x86 where various combinations of the
BIOS, Video ROM and the legacy PC architecture can collude to make life
hard (like not being able to execute the VBIOS twice, or only working in
primary xor secondary mode, accesses which by-pass the SMMU, etc). I
don't know specifically of anything similar with ARM or Mali.

> 
> > (disclaimer: From here on I'm a bit outside the realms of things I'm
> > getting even further from the space where I'm confident in my own
> > understanding...)
> > 
> > Various people have tried to solve this usecase by providing a PV 3D
> > (OpenGL, Gallium, etc) protocol over the PV FB, but I think this turns
> > out to be quite tricky (in terms of shuffling textures around as well as
> > being able to validate the command stream) and generally little comes of
> > it beyond a PoC. (It's been quite some time since I've seen someone try
> > this approach).
> > 
> > The other approach would involve exposing the actual hardware to the
> > guest(s). In general unless your hardware has some special features
> > (vGPU or SR-IOV or something similar) it is rarely safe to let two
> > guests access the same bit of h/w unless they are both willing to
> > co-operate and are assumed to be mutually destructive.
> > 
> > Various folks, including Globalogic who Dario pointed too and Intel with
> > XenGT (which has been renamed but I don't recall the new name) have
> > tried to take a hybrid approach emulating parts of the device in some
> > privileged layer and exposing some parts of the h/w directly to the
> > guest. AIUI this is necessarily specific to a particular GPU device and
> > needs the device to have some convenient properties, but falls short of
> > requiring full "virtualisability" of the h/w ala vGPU or SR-IOV.
> 
> Ok, so I guess that rules out entirely being able to use the GPUs from
> both VMs.

Without significant s/w work, yes.

Ian.

Re: Sharing display between guests

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 5850 bytes --]

On Mon, Jul 06, 2015 at 01:41:34PM +0100, Ian Campbell wrote:
> On Mon, 2015-07-06 at 14:26 +0200, Maxime Ripard wrote:
> > Hi Ian,
> > 
> > On Mon, Jul 06, 2015 at 12:53:49PM +0100, Ian Campbell wrote:
> > > (switching to my work address) 
> > > 
> > > On Thu, 2015-07-02 at 13:13 +0200, Maxime Ripard wrote:
> > > > Hi,
> > > > 
> > > > I've started using Xen on an Allwinner A33, which works great as an
> > > > headless device using the latest PSCI patches in U-Boot.
> > > > 
> > > > However, we would like to do something more with it, and we would need
> > > > to have two VMs accessing the display at once, each one drawing in its
> > > > own part of the framebuffer.
> > > > 
> > > > Something that would look like this:
> > > > 
> > > >    Framebuffer
> > > > +---------------+
> > > > |               |
> > > > |    Guest 1    |
> > > > |               |
> > > > +---------------+
> > > > |               |
> > > > |               |
> > > > |               |
> > > > |    Guest 2    |
> > > > |               |
> > > > |               |
> > > > |               |
> > > > +---------------+
> > > > 
> > > > Where thing start to get interesting is that the second guest would be
> > > > running Android, and as such would need OpenGL support, and access to
> > > > the GPU, and that ideally the first guest would need to be able to
> > > > draw over all the screen to create some kind of a drop-down menu.
> > > > 
> > > > Our first thought was to use two different planes of a DRM/KMS driver,
> > > > one for each VM, with the second guest having the primary plane, and
> > > > the first guest having an overlay, and we would set it up in dom0.
> > > > 
> > > > That would mean that we would have a static "composition", that would
> > > > be setup once and we could forget about it during the life of the
> > > > system.
> > > > 
> > > > This way, we would also have a fixed size framebuffer assigned to
> > > > Android, which is much easier to support, and since we have total
> > > > control over the application in the first guest, we would be able to
> > > > control how much "transparency" we want to leave (== how much of
> > > > Android do we want to be displayed), and we would be able to create
> > > > our drop-down menu.
> > > > 
> > > > Now the hard part: is such a setup possible at all with Xen? Can we
> > > > export a single plane to a guest and let it be the only user of it? If
> > > > that is possible, how would that interact with the 3D acceleration?
> > > 
> > > As it stands today Xen supports exposing a 2D only PV graphics device
> > > (called "pvfb", essentially a flat/linear framebuffer) to guests, I
> > > think this has limited 2D acceleration support let alone 3D support. You
> > > can also (at least in principal, i.e. if the hardware isn't too strange)
> > > pass a GPU through to exactly one guest (often it's just dom0).
> > > 
> > > Some people have also (I think) managed to use pvfb to expose a real
> > > linear fb to a guest, e.g. a "surface" in the real GPU which is setup by
> > > some more privileged domain.
> > > 
> > > But, it sounds like you really want a way to expose the GPU
> > > functionality (i.e. 3D support) to at least one if not both guests.
> > > 
> > > Are you expecting Guest 1 and Guest 2 to be isolated from each other in
> > > a secure way? Or is one considered to be more privileged than the other?
> > 
> > I'd like to have both guests as isolated from each other as possible.
> > 
> > But there's two sub-issues here actually:
> > 
> > 1) Being able to have two VMs showing stuff on the same framebuffer at
> > the same time, without being aware of the other. If we could make pvfb
> > run on a DRM/KMS plane, that would be perfect for us. Another solution
> > would be to split the framebuffer in half and have pvfb running on
> > those two halves.
> 
> Right, this portion is a bit more tractable in the short term I think.
> 
> > 2) Then, one of the guests would be Android, which really requires
> > OpenGL, but the second guests would ideally use it as well, but if it
> > complicates things (and it really seems like it does), we can drop
> > that from the requirement, which would make us fall in the "exactly
> > one guest" case you were talking about (except that it wouldn't dom0,
> > but one of the domU's that would access it).
> 
> IOW give the Android full access to the h/w and have it provide a 2d
> pvfb to the other guest, yes I think that could be made to work.

Not really. I was thinking having the display controller in dom0,
exposing two pvfb, one for Android, one for the other VM, and the GPU
only being passed only to the Android VM.

I don't know whether it makes sense...

> However, there is a wrinkle, which is that passthrough normally requires
> an SMMU, which I'm reasonably sure the A33 doesn't have.

Indeed.

> The usual workaround (which I think Glaballogic use) is to allocate
> guest memory 1:1 (i.e. so it matches in IPA and PA space). This is a
> hack though and we don't like it :-/.

I can imagine :)

> > Our hardware is an A33-based board, that uses a Mali. I'm not exactly
> > sure about what you meant with your "if the hardware isn't too
> > strange". Is there features or hints I should be looking for?
> 
> That was mostly a reference to x86 where various combinations of the
> BIOS, Video ROM and the legacy PC architecture can collude to make life
> hard (like not being able to execute the VBIOS twice, or only working in
> primary xor secondary mode, accesses which by-pass the SMMU, etc). I
> don't know specifically of anything similar with ARM or Mali.

Ok.
 
Thanks!
Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Ian Campbell]

On Mon, 2015-07-06 at 16:10 +0200, Maxime Ripard wrote:
> On Mon, Jul 06, 2015 at 01:41:34PM +0100, Ian Campbell wrote:
> > On Mon, 2015-07-06 at 14:26 +0200, Maxime Ripard wrote:
> > > Hi Ian,
> > > 
> > > On Mon, Jul 06, 2015 at 12:53:49PM +0100, Ian Campbell wrote:
> > > > (switching to my work address) 
> > > > 
> > > > On Thu, 2015-07-02 at 13:13 +0200, Maxime Ripard wrote:
> > > > > Hi,
> > > > > 
> > > > > I've started using Xen on an Allwinner A33, which works great as an
> > > > > headless device using the latest PSCI patches in U-Boot.
> > > > > 
> > > > > However, we would like to do something more with it, and we would need
> > > > > to have two VMs accessing the display at once, each one drawing in its
> > > > > own part of the framebuffer.
> > > > > 
> > > > > Something that would look like this:
> > > > > 
> > > > >    Framebuffer
> > > > > +---------------+
> > > > > |               |
> > > > > |    Guest 1    |
> > > > > |               |
> > > > > +---------------+
> > > > > |               |
> > > > > |               |
> > > > > |               |
> > > > > |    Guest 2    |
> > > > > |               |
> > > > > |               |
> > > > > |               |
> > > > > +---------------+
> > > > > 
> > > > > Where thing start to get interesting is that the second guest would be
> > > > > running Android, and as such would need OpenGL support, and access to
> > > > > the GPU, and that ideally the first guest would need to be able to
> > > > > draw over all the screen to create some kind of a drop-down menu.
> > > > > 
> > > > > Our first thought was to use two different planes of a DRM/KMS driver,
> > > > > one for each VM, with the second guest having the primary plane, and
> > > > > the first guest having an overlay, and we would set it up in dom0.
> > > > > 
> > > > > That would mean that we would have a static "composition", that would
> > > > > be setup once and we could forget about it during the life of the
> > > > > system.
> > > > > 
> > > > > This way, we would also have a fixed size framebuffer assigned to
> > > > > Android, which is much easier to support, and since we have total
> > > > > control over the application in the first guest, we would be able to
> > > > > control how much "transparency" we want to leave (== how much of
> > > > > Android do we want to be displayed), and we would be able to create
> > > > > our drop-down menu.
> > > > > 
> > > > > Now the hard part: is such a setup possible at all with Xen? Can we
> > > > > export a single plane to a guest and let it be the only user of it? If
> > > > > that is possible, how would that interact with the 3D acceleration?
> > > > 
> > > > As it stands today Xen supports exposing a 2D only PV graphics device
> > > > (called "pvfb", essentially a flat/linear framebuffer) to guests, I
> > > > think this has limited 2D acceleration support let alone 3D support. You
> > > > can also (at least in principal, i.e. if the hardware isn't too strange)
> > > > pass a GPU through to exactly one guest (often it's just dom0).
> > > > 
> > > > Some people have also (I think) managed to use pvfb to expose a real
> > > > linear fb to a guest, e.g. a "surface" in the real GPU which is setup by
> > > > some more privileged domain.
> > > > 
> > > > But, it sounds like you really want a way to expose the GPU
> > > > functionality (i.e. 3D support) to at least one if not both guests.
> > > > 
> > > > Are you expecting Guest 1 and Guest 2 to be isolated from each other in
> > > > a secure way? Or is one considered to be more privileged than the other?
> > > 
> > > I'd like to have both guests as isolated from each other as possible.
> > > 
> > > But there's two sub-issues here actually:
> > > 
> > > 1) Being able to have two VMs showing stuff on the same framebuffer at
> > > the same time, without being aware of the other. If we could make pvfb
> > > run on a DRM/KMS plane, that would be perfect for us. Another solution
> > > would be to split the framebuffer in half and have pvfb running on
> > > those two halves.
> > 
> > Right, this portion is a bit more tractable in the short term I think.
> > 
> > > 2) Then, one of the guests would be Android, which really requires
> > > OpenGL, but the second guests would ideally use it as well, but if it
> > > complicates things (and it really seems like it does), we can drop
> > > that from the requirement, which would make us fall in the "exactly
> > > one guest" case you were talking about (except that it wouldn't dom0,
> > > but one of the domU's that would access it).
> > 
> > IOW give the Android full access to the h/w and have it provide a 2d
> > pvfb to the other guest, yes I think that could be made to work.
> 
> Not really. I was thinking having the display controller in dom0,
> exposing two pvfb, one for Android, one for the other VM, and the GPU
> only being passed only to the Android VM.
> 
> I don't know whether it makes sense...

I'm too used to thinking of a graphics device as a single monolithic
whole, rather than as separate/independent display controller and GPU
etc and am not quite comfortable yet with this idea that they might be
independent things which you can mix and match ;-).

To me it seems like the hard bit will likely be the guest side s/w and
whether you can convince it to be happy with a GPU without the display
controller it normally expects but a PVFB instead. But I think that's
pretty much smack bang in the middle of your bailiwick...

Ian.

Re: Sharing display between guests

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1447 bytes --]

On Mon, 2015-07-06 at 13:39 +0200, Maxime Ripard wrote:
> Hi Dario
> 
> On Thu, Jul 02, 2015 at 02:18:18PM +0200, Dario Faggioli wrote:
> > I don't have much comments on all this, just that, reading, it, it
> > sounded somewhat similar to what GL is doing:
> > 
> > http://www.xenproject.org/component/allvideoshare/video/latest/ces15-globalogic.html
> > http://www.xenproject.org/component/allvideoshare/video/globallogic-xen-android.html
> 
> For some reason, the first video wouldn't play here, so I can really
> comment on the second one, but I don't think there was a screen
> sharing here. 
>
Yeah, well, here it is the direct youtube link:

https://www.youtube.com/watch?v=TGiqhG_htXU

I'm "insisting" because the first video is where it's shown that there
are stuff coming from a domain being drown on top of the screen of the
other (when, one is dom0, I think, as I said already).

I'm rather sure they do composition (although, yes, probably in a
different way than you need it), as I've head them talking about it.

But really, that's all I know, so I'll stop here, and defer anything
else to them, if they want to comment. :-)

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 6206 bytes --]

On Mon, Jul 06, 2015 at 03:23:30PM +0100, Ian Campbell wrote:
> On Mon, 2015-07-06 at 16:10 +0200, Maxime Ripard wrote:
> > On Mon, Jul 06, 2015 at 01:41:34PM +0100, Ian Campbell wrote:
> > > On Mon, 2015-07-06 at 14:26 +0200, Maxime Ripard wrote:
> > > > Hi Ian,
> > > > 
> > > > On Mon, Jul 06, 2015 at 12:53:49PM +0100, Ian Campbell wrote:
> > > > > (switching to my work address) 
> > > > > 
> > > > > On Thu, 2015-07-02 at 13:13 +0200, Maxime Ripard wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > I've started using Xen on an Allwinner A33, which works great as an
> > > > > > headless device using the latest PSCI patches in U-Boot.
> > > > > > 
> > > > > > However, we would like to do something more with it, and we would need
> > > > > > to have two VMs accessing the display at once, each one drawing in its
> > > > > > own part of the framebuffer.
> > > > > > 
> > > > > > Something that would look like this:
> > > > > > 
> > > > > >    Framebuffer
> > > > > > +---------------+
> > > > > > |               |
> > > > > > |    Guest 1    |
> > > > > > |               |
> > > > > > +---------------+
> > > > > > |               |
> > > > > > |               |
> > > > > > |               |
> > > > > > |    Guest 2    |
> > > > > > |               |
> > > > > > |               |
> > > > > > |               |
> > > > > > +---------------+
> > > > > > 
> > > > > > Where thing start to get interesting is that the second guest would be
> > > > > > running Android, and as such would need OpenGL support, and access to
> > > > > > the GPU, and that ideally the first guest would need to be able to
> > > > > > draw over all the screen to create some kind of a drop-down menu.
> > > > > > 
> > > > > > Our first thought was to use two different planes of a DRM/KMS driver,
> > > > > > one for each VM, with the second guest having the primary plane, and
> > > > > > the first guest having an overlay, and we would set it up in dom0.
> > > > > > 
> > > > > > That would mean that we would have a static "composition", that would
> > > > > > be setup once and we could forget about it during the life of the
> > > > > > system.
> > > > > > 
> > > > > > This way, we would also have a fixed size framebuffer assigned to
> > > > > > Android, which is much easier to support, and since we have total
> > > > > > control over the application in the first guest, we would be able to
> > > > > > control how much "transparency" we want to leave (== how much of
> > > > > > Android do we want to be displayed), and we would be able to create
> > > > > > our drop-down menu.
> > > > > > 
> > > > > > Now the hard part: is such a setup possible at all with Xen? Can we
> > > > > > export a single plane to a guest and let it be the only user of it? If
> > > > > > that is possible, how would that interact with the 3D acceleration?
> > > > > 
> > > > > As it stands today Xen supports exposing a 2D only PV graphics device
> > > > > (called "pvfb", essentially a flat/linear framebuffer) to guests, I
> > > > > think this has limited 2D acceleration support let alone 3D support. You
> > > > > can also (at least in principal, i.e. if the hardware isn't too strange)
> > > > > pass a GPU through to exactly one guest (often it's just dom0).
> > > > > 
> > > > > Some people have also (I think) managed to use pvfb to expose a real
> > > > > linear fb to a guest, e.g. a "surface" in the real GPU which is setup by
> > > > > some more privileged domain.
> > > > > 
> > > > > But, it sounds like you really want a way to expose the GPU
> > > > > functionality (i.e. 3D support) to at least one if not both guests.
> > > > > 
> > > > > Are you expecting Guest 1 and Guest 2 to be isolated from each other in
> > > > > a secure way? Or is one considered to be more privileged than the other?
> > > > 
> > > > I'd like to have both guests as isolated from each other as possible.
> > > > 
> > > > But there's two sub-issues here actually:
> > > > 
> > > > 1) Being able to have two VMs showing stuff on the same framebuffer at
> > > > the same time, without being aware of the other. If we could make pvfb
> > > > run on a DRM/KMS plane, that would be perfect for us. Another solution
> > > > would be to split the framebuffer in half and have pvfb running on
> > > > those two halves.
> > > 
> > > Right, this portion is a bit more tractable in the short term I think.
> > > 
> > > > 2) Then, one of the guests would be Android, which really requires
> > > > OpenGL, but the second guests would ideally use it as well, but if it
> > > > complicates things (and it really seems like it does), we can drop
> > > > that from the requirement, which would make us fall in the "exactly
> > > > one guest" case you were talking about (except that it wouldn't dom0,
> > > > but one of the domU's that would access it).
> > > 
> > > IOW give the Android full access to the h/w and have it provide a 2d
> > > pvfb to the other guest, yes I think that could be made to work.
> > 
> > Not really. I was thinking having the display controller in dom0,
> > exposing two pvfb, one for Android, one for the other VM, and the GPU
> > only being passed only to the Android VM.
> > 
> > I don't know whether it makes sense...
> 
> I'm too used to thinking of a graphics device as a single monolithic
> whole, rather than as separate/independent display controller and GPU
> etc and am not quite comfortable yet with this idea that they might be
> independent things which you can mix and match ;-).

Note that I'm not saying it would work though. It's just a thought
from someone not-so-experienced with GPUs and virtualization :)

> To me it seems like the hard bit will likely be the guest side s/w and
> whether you can convince it to be happy with a GPU without the display
> controller it normally expects but a PVFB instead. But I think that's
> pretty much smack bang in the middle of your bailiwick...

That's what I thought, and the reason of me starting this whole
discussion..

Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Sharing display between guests

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 1467 bytes --]

On Mon, Jul 06, 2015 at 05:14:54PM +0200, Dario Faggioli wrote:
> On Mon, 2015-07-06 at 13:39 +0200, Maxime Ripard wrote:
> > Hi Dario
> > 
> > On Thu, Jul 02, 2015 at 02:18:18PM +0200, Dario Faggioli wrote:
> > > I don't have much comments on all this, just that, reading, it, it
> > > sounded somewhat similar to what GL is doing:
> > > 
> > > http://www.xenproject.org/component/allvideoshare/video/latest/ces15-globalogic.html
> > > http://www.xenproject.org/component/allvideoshare/video/globallogic-xen-android.html
> > 
> > For some reason, the first video wouldn't play here, so I can really
> > comment on the second one, but I don't think there was a screen
> > sharing here. 
> >
> Yeah, well, here it is the direct youtube link:
> 
> https://www.youtube.com/watch?v=TGiqhG_htXU
> 
> I'm "insisting" because the first video is where it's shown that there
> are stuff coming from a domain being drown on top of the screen of the
> other (when, one is dom0, I think, as I said already).
> 
> I'm rather sure they do composition (although, yes, probably in a
> different way than you need it), as I've head them talking about it.

Ok, I first though that it was two different VMs, displaying stuff on
two different displays, which makes things a bit easier :)

My bad then, and thanks for pointing this out!
Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel