about the current status of Multi-process QEMU / out-of-process emulation

about the current status of Multi-process QEMU / out-of-process emulation

[Yu Zhang]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

Hi All,

I saw that you authored the QEMU page for "Multi-process QEMU". (
https://www.qemu.org/docs/master/system/multi-process.html)

I'm interested in this feature, but feel a little confused with the command
line:

+      /usr/bin/qemu-system-x86_64                                        \
+      -machine x-remote                                                  \
+      -device lsi53c895a,id=lsi0                                         \
+      -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2           \
+      -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0  \
+      -object x-remote-object,id=robj1,devid=lsi1,fd=4,
(https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg02887.html)

It seems that the man page of qemu command contains no parameter and option
yet for this feature. May I know whether is it still in experimental
stage? And even a few more questions:

- Is "x-remote" a standalone machine type for creating the orchestrator?
- Can each device has a dedicated emulation process or shares one process
for emulating multiple devices?
- Are there more command line examples illustrating the combination of
orchestrator, remote emulation process, memory-backend-memfd and
x-pci-proxy-dev?

Thank you very much
Kind regard

Yu Zhang @ IONOS Compute Platform
03.06.2022

[-- Attachment #2: Type: text/html, Size: 1769 bytes --]

Fwd: about the current status of Multi-process QEMU / out-of-process emulation

[Yu Zhang]

[-- Attachment #1: Type: text/plain, Size: 3609 bytes --]

Hello Dongli, Elena, John, and Jagannathan,

I'm interested in the "multi-process QEMU" feature and got the kind reply
by Mr. Vivier that I may contact you for this.
On one of the QEMU docs [1] I saw the command line:

+      /usr/bin/qemu-system-x86_64                                        \
+      -machine x-remote                                                  \
+      -device lsi53c895a,id=lsi0                                         \
+      -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2           \
+      -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0  \
+      -object x-remote-object,id=robj1,devid=lsi1,fd=4,

It seems that the man page of qemu contains no parameter and option yet for
this feature. The qemu docs, such as [2][3][4] are either not up-to-date or
"doesn't reflect the current status of the implementation".
So may I know whether is it still in experimental stage or mature enough
for use? And even a few further questions:

- When creating the orchestrator, can we specify a machine type such as
pc-i440fx-7.0 for -machine?
- Can each device has a dedicated emulation process or shares one process
for emulating multiple devices?
- Can we find more command line examples showing the combination of
orchestrator, remote emulation process, memory-backend-memfd and
x-pci-proxy-dev?

Thank you very much and all the best

Yu Zhang
03.06.2022

[1] https://www.qemu.org/docs/master/system/multi-process.html
[2] https://wiki.qemu.org/Features/MultiProcessQEMU
[3]
https://lxr.missinglinkelectronics.com/qemu+v7.0.0/docs/devel/multi-process.rst
[4] https://qemu.readthedocs.io/en/latest/devel/multi-process.html

---------- Forwarded message ---------
From: Laurent Vivier <laurent@vivier.eu>
Date: Fri, Jun 3, 2022 at 4:14 PM
Subject: Re: about the current status of Multi-process QEMU /
out-of-process emulation
To: Yu Zhang <yu.zhang@ionos.com>


Hi Yu,

I'm not the author of this documentation, only the person that has merged
the last change in the repo.

According to the logs you should contact Dongli Zhang <
dongli.zhang@oracle.com>, Elena Ufimtseva
<elena.ufimtseva@oracle.com>, John G Johnson <john.g.johnson@oracle.com> or
Jagannathan Raman
<jag.raman@oracle.com> .

Thanks,
Laurent

Le 03/06/2022 à 12:17, Yu Zhang a écrit :
> Dear Mr. Vivier,
>
> I saw that you authored the QEMU page for "Multi-process QEMU".
> (https://www.qemu.org/docs/master/system/multi-process.html
> <https://www.qemu.org/docs/master/system/multi-process.html>)
>
> I'm interested in this feature, but feel a little confused with the
command line:
>
> +      /usr/bin/qemu-system-x86_64
 \
> +      -machine x-remote
 \
> +      -device lsi53c895a,id=lsi0
\
> +      -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2
\
> +      -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0
 \
> +      -object x-remote-object,id=robj1,devid=lsi1,fd=4,
>
> It seems that the man page of qemu command contains no parameter and
option yet for this feature.
> May I know whether is it still in experimental stage? And even a few more
questions:
>
> - Is "x-remote" a standalone machine type for creating the orchestrator?
> - Can each device has a dedicated emulation process or shares one process
for emulating multiple
> devices?
> - Can I find more command line examples illustrating the combination of
orchestrator, remote
> emulation process, memory-backend-memfd and x-pci-proxy-dev?
>
> Thank you very much
> Kind regard
>
> Yu Zhang
> 03.06.2022

[-- Attachment #2: Type: text/html, Size: 5440 bytes --]

Re: about the current status of Multi-process QEMU / out-of-process emulation

[Jag Raman]

[-- Attachment #1: Type: text/plain, Size: 5056 bytes --]



On Jun 3, 2022, at 11:34 AM, Yu Zhang <yu.zhang@ionos.com<mailto:yu.zhang@ionos.com>> wrote:

Hello Dongli, Elena, John, and Jagannathan,

I'm interested in the "multi-process QEMU" feature and got the kind reply by Mr. Vivier that I may contact you for this.
On one of the QEMU docs [1] I saw the command line:

+      /usr/bin/qemu-system-x86_64                                        \
+      -machine x-remote                                                  \
+      -device lsi53c895a,id=lsi0                                         \
+      -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2           \
+      -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0  \
+      -object x-remote-object,id=robj1,devid=lsi1,fd=4,

It seems that the man page of qemu contains no parameter and option yet for this feature. The qemu docs, such as [2][3][4] are either not up-to-date or "doesn't reflect the current status of the implementation".
So may I know whether is it still in experimental stage or mature enough for use? And even a few further questions:

Hello Yu,

We are working on vfio-user for QEMU which would supersede multi-process QEMU.
The vfio-user feature is currently under review for merging with QEMU. We would drop
multi-process QEMU support after the vfio-user changes are merged.

We use the following repo for testing vfio-user before sending the patches for
review. You may be interested in checking it out:
https://github.com/oracle/qemu/tree/master


- When creating the orchestrator, can we specify a machine type such as pc-i440fx-7.0 for -machine?

For vfio-user, the machine type on the remote QEMU process (server) is
always “x-remote”. The client QEMU could be of any machine type.

- Can each device has a dedicated emulation process or shares one process for emulating multiple devices?

Each device could be running in a dedicated process, or multiple
devices could share one process.

- Can we find more command line examples showing the combination of orchestrator, remote emulation process, memory-backend-memfd and x-pci-proxy-dev?

For vfio-user, we could give you a heads up once they are merged into QEMU. We
are using the following for testing our changes, which you could checkout
in the meanwhile:
scripts/vfiouser-launcher.py

Could you please give us more details about what you’re trying to do? Which
devices are you trying to emulate in the remote process?

Thank you!
--
Jag


Thank you very much and all the best

Yu Zhang
03.06.2022

[1] https://www.qemu.org/docs/master/system/multi-process.html
[2] https://wiki.qemu.org/Features/MultiProcessQEMU
[3] https://lxr.missinglinkelectronics.com/qemu+v7.0.0/docs/devel/multi-process.rst
[4] https://qemu.readthedocs.io/en/latest/devel/multi-process.html

---------- Forwarded message ---------
From: Laurent Vivier <laurent@vivier.eu<mailto:laurent@vivier.eu>>
Date: Fri, Jun 3, 2022 at 4:14 PM
Subject: Re: about the current status of Multi-process QEMU / out-of-process emulation
To: Yu Zhang <yu.zhang@ionos.com<mailto:yu.zhang@ionos.com>>


Hi Yu,

I'm not the author of this documentation, only the person that has merged the last change in the repo.

According to the logs you should contact Dongli Zhang <dongli.zhang@oracle.com<mailto:dongli.zhang@oracle.com>>, Elena Ufimtseva
<elena.ufimtseva@oracle.com<mailto:elena.ufimtseva@oracle.com>>, John G Johnson <john.g.johnson@oracle.com<mailto:john.g.johnson@oracle.com>> or Jagannathan Raman
<jag.raman@oracle.com<mailto:jag.raman@oracle.com>> .

Thanks,
Laurent

Le 03/06/2022 à 12:17, Yu Zhang a écrit :
> Dear Mr. Vivier,
>
> I saw that you authored the QEMU page for "Multi-process QEMU".
> (https://www.qemu.org/docs/master/system/multi-process.html
> <https://www.qemu.org/docs/master/system/multi-process.html>)
>
> I'm interested in this feature, but feel a little confused with the command line:
>
> +      /usr/bin/qemu-system-x86_64                                        \
> +      -machine x-remote                                                  \
> +      -device lsi53c895a,id=lsi0                                         \
> +      -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2           \
> +      -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0  \
> +      -object x-remote-object,id=robj1,devid=lsi1,fd=4,
>
> It seems that the man page of qemu command contains no parameter and option yet for this feature.
> May I know whether is it still in experimental stage? And even a few more questions:
>
> - Is "x-remote" a standalone machine type for creating the orchestrator?
> - Can each device has a dedicated emulation process or shares one process for emulating multiple
> devices?
> - Can I find more command line examples illustrating the combination of orchestrator, remote
> emulation process, memory-backend-memfd and x-pci-proxy-dev?
>
> Thank you very much
> Kind regard
>
> Yu Zhang
> 03.06.2022



[-- Attachment #2: Type: text/html, Size: 9853 bytes --]

Re: about the current status of Multi-process QEMU / out-of-process emulation

[Yu Zhang]

[-- Attachment #1: Type: text/plain, Size: 6190 bytes --]

Hello Jag,

It's quite nice of you to give me so much details about this feature. I
tried with this repo by using the python script. The server and client qemu
processes are created successfully.

I'm interested in it due to the requirement for higher security level in
cloud environment. We're taking efforts to enhance the security of the VMs
and hosts.

I read on the page below

   https://www.qemu.org/docs/master/devel/multi-process.html

that "an attacker who compromised this service would not be able to use
this exploit to access files or devices beyond what the disk service was
given access to". This is the feature which can reduce the risk due to a
compromised device emulation code.

It doesn't matter which specific device can be emulated in a separated
process. It matters that the device emulation service can be separated from
the qemu main process.

Another aspect I'd like to know is, could the multi-processes be live
migrated just as the single qemu process?

Thank you so much for your time and patience.
Wish you all the best,

Yu Zhang
07.06.2022

On Fri, Jun 3, 2022 at 7:37 PM Jag Raman <jag.raman@oracle.com> wrote:

>
>
> On Jun 3, 2022, at 11:34 AM, Yu Zhang <yu.zhang@ionos.com> wrote:
>
> Hello Dongli, Elena, John, and Jagannathan,
>
> I'm interested in the "multi-process QEMU" feature and got the kind reply
> by Mr. Vivier that I may contact you for this.
> On one of the QEMU docs [1] I saw the command line:
>
> +      /usr/bin/qemu-system-x86_64                                        \
> +      -machine x-remote                                                  \
> +      -device lsi53c895a,id=lsi0                                         \
> +      -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2           \
> +      -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0  \
> +      -object x-remote-object,id=robj1,devid=lsi1,fd=4,
>
> It seems that the man page of qemu contains no parameter and option yet
> for this feature. The qemu docs, such as [2][3][4] are either not
> up-to-date or "doesn't reflect the current status of the implementation".
> So may I know whether is it still in experimental stage or mature enough
> for use? And even a few further questions:
>
>
> Hello Yu,
>
> We are working on vfio-user for QEMU which would supersede multi-process
> QEMU.
> The vfio-user feature is currently under review for merging with QEMU. We
> would drop
> multi-process QEMU support after the vfio-user changes are merged.
>
> We use the following repo for testing vfio-user before sending the patches
> for
> review. You may be interested in checking it out:
> https://github.com/oracle/qemu/tree/master
>
>
> - When creating the orchestrator, can we specify a machine type such as
> pc-i440fx-7.0 for -machine?
>
>
> For vfio-user, the machine type on the remote QEMU process (server) is
> always “x-remote”. The client QEMU could be of any machine type.
>
> - Can each device has a dedicated emulation process or shares one process
> for emulating multiple devices?
>
>
> Each device could be running in a dedicated process, or multiple
> devices could share one process.
>
> - Can we find more command line examples showing the combination of
> orchestrator, remote emulation process, memory-backend-memfd and
> x-pci-proxy-dev?
>
>
> For vfio-user, we could give you a heads up once they are merged into
> QEMU. We
> are using the following for testing our changes, which you could checkout
> in the meanwhile:
> scripts/vfiouser-launcher.py
>
> Could you please give us more details about what you’re trying to do? Which
> devices are you trying to emulate in the remote process?
>
> Thank you!
> --
> Jag
>
>
> Thank you very much and all the best
>
> Yu Zhang
> 03.06.2022
>
> [1] https://www.qemu.org/docs/master/system/multi-process.html
> [2] https://wiki.qemu.org/Features/MultiProcessQEMU
> [3]
> https://lxr.missinglinkelectronics.com/qemu+v7.0.0/docs/devel/multi-process.rst
> [4] https://qemu.readthedocs.io/en/latest/devel/multi-process.html
>
> ---------- Forwarded message ---------
> From: Laurent Vivier <laurent@vivier.eu>
> Date: Fri, Jun 3, 2022 at 4:14 PM
> Subject: Re: about the current status of Multi-process QEMU /
> out-of-process emulation
> To: Yu Zhang <yu.zhang@ionos.com>
>
>
> Hi Yu,
>
> I'm not the author of this documentation, only the person that has merged
> the last change in the repo.
>
> According to the logs you should contact Dongli Zhang <
> dongli.zhang@oracle.com>, Elena Ufimtseva
> <elena.ufimtseva@oracle.com>, John G Johnson <john.g.johnson@oracle.com>
> or Jagannathan Raman
> <jag.raman@oracle.com> .
>
> Thanks,
> Laurent
>
> Le 03/06/2022 à 12:17, Yu Zhang a écrit :
> > Dear Mr. Vivier,
> >
> > I saw that you authored the QEMU page for "Multi-process QEMU".
> > (https://www.qemu.org/docs/master/system/multi-process.html
> > <https://www.qemu.org/docs/master/system/multi-process.html>)
> >
> > I'm interested in this feature, but feel a little confused with the
> command line:
> >
> > +      /usr/bin/qemu-system-x86_64
>  \
> > +      -machine x-remote
>  \
> > +      -device lsi53c895a,id=lsi0
>   \
> > +      -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2
>   \
> > +      -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0
>  \
> > +      -object x-remote-object,id=robj1,devid=lsi1,fd=4,
> >
> > It seems that the man page of qemu command contains no parameter and
> option yet for this feature.
> > May I know whether is it still in experimental stage? And even a few
> more questions:
> >
> > - Is "x-remote" a standalone machine type for creating the orchestrator?
> > - Can each device has a dedicated emulation process or shares one
> process for emulating multiple
> > devices?
> > - Can I find more command line examples illustrating the combination of
> orchestrator, remote
> > emulation process, memory-backend-memfd and x-pci-proxy-dev?
> >
> > Thank you very much
> > Kind regard
> >
> > Yu Zhang
> > 03.06.2022
>
>
>

[-- Attachment #2: Type: text/html, Size: 9452 bytes --]