[Qemu-devel] Proof of concept for GPU forwarding for Linux guest on Linux host

[Qemu-devel] Proof of concept for GPU forwarding for Linux guest on Linux host

[Tao Wu]

Hi,

This is a proof of concept of GPU forwarding for Linux guest on Linux host.
I'd like to get comments and suggestions from community before I put more
time on it. To summarize what it is:

1. It's a solution to bring GPU acceleration for Linux vm guest on Linux host.
It could works with different GPU although the current proof of concept only
works with Intel GPU.

2. This shows as a kernel patch. The actual qemu diff is nested in 
tools/forward/qemu.diff

3. The basic idea is: under Linux, most applications use GPU acceleration with
help of MESA library. And MESA library interacts with kernel GPU driver by
operating on some special character device file exported by kernel GPU driver.
MESA library opens some special files in system and operations on GPU are done
by ioctl/mmap system call and regular memory operations.

We just write a kernel driver for guest Linux kernel and let it exports same
interface to user space like the real Linux GPU kernel driver. So it's an API proxy
between host and VM guest. We just proxy API at system call level. 

4. Some nasty things was done in guest kernel as a quick dirty hack so we don't need
to touch user space (wayland/mesa etc) now.

5. You can check tools/forward/README for instructions.

Thanks!

Tao Wu (1):
  proof of concept for GPU forwarding

 arch/x86/configs/x86_64_defconfig      |    5 +
 drivers/char/Makefile                  |    1 +
 drivers/char/forwarder/Makefile        |    8 +
 drivers/char/forwarder/forwarder.h     |  103 ++
 drivers/char/forwarder/forwarder_drv.c | 2104 ++++++++++++++++++++++++
 fs/open.c                              |   18 +
 include/uapi/linux/virtwl.h            |   64 +
 tools/forward/Makefile                 |    2 +
 tools/forward/README                   |   58 +
 tools/forward/qemu.diff                | 1117 +++++++++++++
 tools/forward/wayland-proxy-main.c     |   58 +
 tools/forward/wayland-proxy.c          |  297 ++++
 12 files changed, 3835 insertions(+)
 create mode 100644 drivers/char/forwarder/Makefile
 create mode 100644 drivers/char/forwarder/forwarder.h
 create mode 100644 drivers/char/forwarder/forwarder_drv.c
 create mode 100644 include/uapi/linux/virtwl.h
 create mode 100644 tools/forward/Makefile
 create mode 100644 tools/forward/README
 create mode 100644 tools/forward/qemu.diff
 create mode 100644 tools/forward/wayland-proxy-main.c
 create mode 100644 tools/forward/wayland-proxy.c

-- 
2.21.0.392.gf8f6787159e-goog

Re: [Qemu-devel] Proof of concept for GPU forwarding for Linux guest on Linux host

[Andrew Randrianasulu]

Hi!

I recall I saw something like this some time ago (but in connection with qemu-user + ALSA sycallls)
So far I found this:

https://marc.info/?l=qemu-devel&m=154349285205915

List:       qemu-devel
Subject:    [Qemu-devel] Support for Intel GPU
From:       Allan Sandfeld Jensen <kde () carewolf ! com>
Date:       2018-11-29 11:54:19

Ah, found alsa bits too:
https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg05334.html

None of this was merged, as far as I understand.

Re: [Qemu-devel] Proof of concept for GPU forwarding for Linux guest on Linux host

[Tao Wu(吴涛@Eng)]

On Thu, Apr 4, 2019 at 12:15 AM Andrew Randrianasulu <
randrianasulu@gmail.com> wrote:

> Hi!
>
> I recall I saw something like this some time ago (but in connection with
> qemu-user + ALSA sycallls)
> So far I found this:
>
> https://marc.info/?l=qemu-devel&m=154349285205915
>
> List:       qemu-devel
> Subject:    [Qemu-devel] Support for Intel GPU
> From:       Allan Sandfeld Jensen <kde () carewolf ! com>
> Date:       2018-11-29 11:54:19
>
> Ah, found alsa bits too:
> https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg05334.html

Thanks, I took a look, the previous patches are targeting qemu user mode
while mine is targeting full system virtualization.
There is no plan to merge my code as it is now. I just want to get some
feedbacks first.


>
>
> None of this was merged, as far as I understand.
>

Re: [Qemu-devel] Proof of concept for GPU forwarding for Linux guest on Linux host

[Andrew Randrianasulu]

В сообщении от Thursday 04 April 2019 10:33:55 Tao Wu(吴涛@Eng) написал(а):
> On Thu, Apr 4, 2019 at 12:15 AM Andrew Randrianasulu <
> randrianasulu@gmail.com> wrote:
> 
> > Hi!
> >
> > I recall I saw something like this some time ago (but in connection with
> > qemu-user + ALSA sycallls)
> > So far I found this:
> >
> > https://marc.info/?l=qemu-devel&m=154349285205915
> >
> > List:       qemu-devel
> > Subject:    [Qemu-devel] Support for Intel GPU
> > From:       Allan Sandfeld Jensen <kde () carewolf ! com>
> > Date:       2018-11-29 11:54:19
> >
> > Ah, found alsa bits too:
> > https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg05334.html
> 
> Thanks, I took a look, the previous patches are targeting qemu user mode
> while mine is targeting full system virtualization.
> There is no plan to merge my code as it is now. I just want to get some
> feedbacks first.

I was thinking qemu's system mode was covered already by

https://www.kraxel.org/blog/2018/04/vgpu-display-support-finally-merged-upstream/
(intel's vGPU)

https://www.kraxel.org/blog/2016/09/using-virtio-gpu-with-libvirt-and-spice/
(virtio-gpu, via gallium framework)

?

But thanks anyway, may be some of those ideas can be fused together (I only have nvidia/nouveau GPU, 
and at best can forward-port something simple)


> 
> 
> >
> >
> > None of this was merged, as far as I understand.
> >
> 

Re: [Qemu-devel] Proof of concept for GPU forwarding for Linux guest on Linux host

[Tao Wu(吴涛@Eng)]

On Thu, Apr 4, 2019 at 12:49 AM Andrew Randrianasulu <
randrianasulu@gmail.com> wrote:

> В сообщении от Thursday 04 April 2019 10:33:55 Tao Wu(吴涛@Eng) написал(а):
> > On Thu, Apr 4, 2019 at 12:15 AM Andrew Randrianasulu <
> > randrianasulu@gmail.com> wrote:
> >
> > > Hi!
> > >
> > > I recall I saw something like this some time ago (but in connection
> with
> > > qemu-user + ALSA sycallls)
> > > So far I found this:
> > >
> > > https://marc.info/?l=qemu-devel&m=154349285205915
> > >
> > > List:       qemu-devel
> > > Subject:    [Qemu-devel] Support for Intel GPU
> > > From:       Allan Sandfeld Jensen <kde () carewolf ! com>
> > > Date:       2018-11-29 11:54:19
> > >
> > > Ah, found alsa bits too:
> > > https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg05334.html
> >
> > Thanks, I took a look, the previous patches are targeting qemu user mode
> > while mine is targeting full system virtualization.
> > There is no plan to merge my code as it is now. I just want to get some
> > feedbacks first.
>
> I was thinking qemu's system mode was covered already by
>
>
> https://www.kraxel.org/blog/2018/04/vgpu-display-support-finally-merged-upstream/
> (intel's vGPU)
>
> Personally I have never tested this.

>
> https://www.kraxel.org/blog/2016/09/using-virtio-gpu-with-libvirt-and-spice/
> (virtio-gpu
> <https://www.kraxel.org/blog/2016/09/using-virtio-gpu-with-libvirt-and-spice/(virtio-gpu>,
> via gallium framework)
>
In the long term, this should be the right direction. But for now, it is
lacking of something like vulkan support.
Also, performance is not as great as it could be. In some use case
(benchmark), GPU forwarding work much better
than virtio-gpu and can get close native performance.

>
> ?
>
> But thanks anyway, may be some of those ideas can be fused together (I
> only have nvidia/nouveau GPU,
> and at best can forward-port something simple)

Actually although I said GPU forwarding, the real code is doing "file
handler" forwarding (by supporting
mmap/ioctl/select...) etc. I am not sure if we should have a general frame
work for things like this so we can forward
any special device in host to guest with minimal efforts.

>






>
> >
> >
> > >
> > >
> > > None of this was merged, as far as I understand.
> > >
> >
>
>
>

Re: [Qemu-devel] Proof of concept for GPU forwarding for Linux guest on Linux host

[Dr. David Alan Gilbert]

cc'ing in Gerd and Marc-André

* Andrew Randrianasulu (randrianasulu@gmail.com) wrote:
> В сообщении от Thursday 04 April 2019 10:33:55 Tao Wu(吴涛@Eng) написал(а):
> > On Thu, Apr 4, 2019 at 12:15 AM Andrew Randrianasulu <
> > randrianasulu@gmail.com> wrote:
> > 
> > > Hi!
> > >
> > > I recall I saw something like this some time ago (but in connection with
> > > qemu-user + ALSA sycallls)
> > > So far I found this:
> > >
> > > https://marc.info/?l=qemu-devel&m=154349285205915
> > >
> > > List:       qemu-devel
> > > Subject:    [Qemu-devel] Support for Intel GPU
> > > From:       Allan Sandfeld Jensen <kde () carewolf ! com>
> > > Date:       2018-11-29 11:54:19
> > >
> > > Ah, found alsa bits too:
> > > https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg05334.html
> > 
> > Thanks, I took a look, the previous patches are targeting qemu user mode
> > while mine is targeting full system virtualization.
> > There is no plan to merge my code as it is now. I just want to get some
> > feedbacks first.
> 
> I was thinking qemu's system mode was covered already by
> 
> https://www.kraxel.org/blog/2018/04/vgpu-display-support-finally-merged-upstream/
> (intel's vGPU)
> 
> https://www.kraxel.org/blog/2016/09/using-virtio-gpu-with-libvirt-and-spice/
> (virtio-gpu, via gallium framework)
> 
> ?
> 
> But thanks anyway, may be some of those ideas can be fused together (I only have nvidia/nouveau GPU, 
> and at best can forward-port something simple)
> 
> 
> > 
> > 
> > >
> > >
> > > None of this was merged, as far as I understand.
> > >
> > 
> 
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK