Re: [PATCH v5 0/6] Add vmnet.framework based network backend

[Vladislav Yaroshchuk <yaroshchuk2000@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 8804 bytes --]

вт, 16 нояб. 2021 г. в 10:23, Jason Wang <jasowang@redhat.com>:

> On Mon, Nov 15, 2021 at 6:45 PM Vladislav Yaroshchuk
> <yaroshchuk2000@gmail.com> wrote:
> >
> >
> >
> > пн, 15 нояб. 2021 г. в 07:53, Jason Wang <jasowang@redhat.com>:
> >>
> >> On Fri, Nov 12, 2021 at 5:14 PM Vladislav Yaroshchuk
> >> <yaroshchuk2000@gmail.com> wrote:
> >> >
> >> > macOS provides networking API for VMs called 'vmnet.framework':
> >> > https://developer.apple.com/documentation/vmnet
> >> >
> >> > We can provide its support as the new QEMU network backends which
> >> > represent three different vmnet.framework interface usage modes:
> >> >
> >> >   * `vmnet-shared`:
> >> >     allows the guest to communicate with other guests in shared mode
> and
> >> >     also with external network (Internet) via NAT. Has
> (macOS-provided)
> >> >     DHCP server; subnet mask and IP range can be configured;
> >> >
> >> >   * `vmnet-host`:
> >> >     allows the guest to communicate with other guests in host mode.
> >> >     By default has enabled DHCP as `vmnet-shared`, but providing
> >> >     network unique id (uuid) can make `vmnet-host` interfaces isolated
> >> >     from each other and also disables DHCP.
> >> >
> >> >   * `vmnet-bridged`:
> >> >     bridges the guest with a physical network interface.
> >> >
> >> > This backends cannot work on macOS Catalina 10.15 cause we use
> >> > vmnet.framework API provided only with macOS 11 and newer. Seems
> >> > that it is not a problem, because QEMU guarantees to work on two most
> >> > recent versions of macOS which now are Big Sur (11) and Monterey (12).
> >> >
> >> > Also, we have one inconvenient restriction: vmnet.framework interfaces
> >> > can create only privileged user:
> >> > `$ sudo qemu-system-x86_64 -nic vmnet-shared`
> >> >
> >> > Attempt of `vmnet-*` netdev creation being unprivileged user fails
> with
> >> > vmnet's 'general failure'.
> >> >
> >> > This happens because vmnet.framework requires
> `com.apple.vm.networking`
> >> > entitlement which is: "restricted to developers of virtualization
> software.
> >> > To request this entitlement, contact your Apple representative." as
> Apple
> >> > documentation says:
> >> >
> https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
> >>
> >> Do you know how multipass work? Looks like it uses vmnet without
> privileges.
> >>
> >
> > I've just checked this, and they still need root privileges. They have a
> > `multipassd` daemon which is launched as root by launchd by default.
> >
> > ```
> > bash-5.1$ ps axo ruser,ruid,comm | grep multipass
> > root                 0 /Library/Application
> Support/com.canonical.multipass/bin/multipassd
> > root                 0 /Library/Application
> Support/com.canonical.multipass/bin/hyperkit
> > ```
> >
> > That's the reason why it's required to 'enter a password' while
> multipass installation:
> > it creates launchd plist (kinda launch rule) and places it to
> /Library/LaunchDaemons/,
> > which can be done only with root privileges.
> >
> > ```
> > bash-5.1$ ll /Library/LaunchDaemons | grep multipass
> > -rw-r--r--   1 root  wheel   1.1K 15 Nov 12:47
> com.canonical.multipassd.plist
> > ```
> >
> > And after this launchd launches this multipassd daemon as root every
> boot.
> >
> > So an unprivileged user can launch a multipass VM instance, but actually
> the
> > `hyperkit` process which interacts with vmnet is gonna be launched by
> multipassd
> > running as root.
>
> I wonder how it passes the vmnet object to qemu? Nothing obvious from
> the qemu command line that multipass launched:
>
> -nic vmnet-macos,mode=shared,model=virtio-net-pci,mac=52:54:00:52:e8:e4
>
> (But I haven't had time to check their qemu codes).
>
>
It seems they just use QEMU with patch by Phillip Tennen:
https://patchew.org/QEMU/20210218134947.1860-1-phillip.ennen@gmail.com/

In that patch he does quite the same as we in this series, the
difference remains in foreground: he creates one new 'vmnet-macos'
netdev, and uses 'mode=shared' property to choose vmnet
operating mode. I decided to create three different netdevs instead
(vmnet-shared, vmnet-host, vmnet-bridged). Also I've added some
features related to isolation and ipv6.

> I wonder how it passes the vmnet object to qemu
I hope I clearly described this.

>
> > tl;dr sadly, we can't interact with vmnet.framework without having our
> binary correctly
> > signed and being an unprivileged user. Root privileges or special
> signature with
> > entitlement is required.
>
> This is something similar to what happens in other OS. E.g for the tap
> backend, it can't be created without privileges. So qemu allows to:
>
> 1) the TAP to be created by privileged program like libvirt, and its
> fd could be passed to qemu via SCM_RIGHTS
> 2) run a set-uid helper to create and config TAP
>
> This is something we need to consider now or in the future probably.
>
>
Seems we can do nothing with this if we have qemu-bundled &
direct vmnet.framework interaction, it always requires privileges
or entitlement.
The workaround can be moving vmnet-related things to
another helper binary running with privileges, and usage of
this helper somewhere between qemu and vmnet.

I think for now it's applicable to leave it as is, having qemu
that requires privileges for vmnet.framework usage.


> Thanks
>
>
> >
> >> Thanks
> >>
> >
> > Thank you for your review, I will check it this week and reply as soon
> as possible.
> >
> >>
> >> >
> >> > One more note: we still have quite useful but not supported
> >> > 'vmnet.framework' features as creating port forwarding rules, IPv6
> >> > NAT prefix specifying and so on.
> >> >
> >> > Nevertheless, new backends work fine and tested within
> `qemu-system-x86-64`
> >> > on macOS Bir Sur 11.5.2 host with such nic models:
> >> >   * e1000-82545em
> >> >   * virtio-net-pci
> >> >   * vmxnet3
> >> >
> >> > The guests were:
> >> >   * macOS 10.15.7
> >> >   * Ubuntu Bionic (server cloudimg)
> >> >
> >> >
> >> > This series partially reuses patches by Phillip Tennen:
> >> >
> https://patchew.org/QEMU/20210218134947.1860-1-phillip.ennen@gmail.com/
> >> > So I included his signed-off line into one of the commit messages and
> >> > also here.
> >> >
> >> > v1 -> v2:
> >> >  Since v1 minor typos were fixed, patches rebased onto latest master,
> >> >  redundant changes removed (small commits squashed)
> >> >
> >> > v2 -> v3:
> >> >  - QAPI style fixes
> >> >  - Typos fixes in comments
> >> >  - `#include`'s updated to be in sync with recent master
> >> > v3 -> v4:
> >> >  - Support vmnet interfaces isolation feature
> >> >  - Support vmnet-host network uuid setting feature
> >> >  - Refactored sources a bit
> >> > v4 -> v5:
> >> >  - Missed 6.2 boat, now 7.0 candidate
> >> >  - Fix qapi netdev descriptions and styles
> >> >    (@subnetmask -> @subnet-mask)
> >> >  - Support vmnet-shared IPv6 prefix setting feature
> >> >
> >> > Signed-off-by: Phillip Tennen <phillip@axleos.com>
> >> > Signed-off-by: Vladislav Yaroshchuk <yaroshchuk2000@gmail.com>
> >> >
> >> > Vladislav Yaroshchuk (6):
> >> >   net/vmnet: add vmnet dependency and customizable option
> >> >   net/vmnet: add vmnet backends to qapi/net
> >> >   net/vmnet: implement shared mode (vmnet-shared)
> >> >   net/vmnet: implement host mode (vmnet-host)
> >> >   net/vmnet: implement bridged mode (vmnet-bridged)
> >> >   net/vmnet: update qemu-options.hx
> >> >
> >> >  meson.build                   |   4 +
> >> >  meson_options.txt             |   2 +
> >> >  net/clients.h                 |  11 ++
> >> >  net/meson.build               |   7 +
> >> >  net/net.c                     |  10 ++
> >> >  net/vmnet-bridged.m           | 111 ++++++++++++
> >> >  net/vmnet-common.m            | 325
> ++++++++++++++++++++++++++++++++++
> >> >  net/vmnet-host.c              | 111 ++++++++++++
> >> >  net/vmnet-shared.c            |  92 ++++++++++
> >> >  net/vmnet_int.h               |  48 +++++
> >> >  qapi/net.json                 | 127 ++++++++++++-
> >> >  qemu-options.hx               |  25 +++
> >> >  scripts/meson-buildoptions.sh |   3 +
> >> >  13 files changed, 874 insertions(+), 2 deletions(-)
> >> >  create mode 100644 net/vmnet-bridged.m
> >> >  create mode 100644 net/vmnet-common.m
> >> >  create mode 100644 net/vmnet-host.c
> >> >  create mode 100644 net/vmnet-shared.c
> >> >  create mode 100644 net/vmnet_int.h
> >> >
> >> > --
> >> > 2.23.0
> >> >
> >>
> >
> >
> > --
> > Best Regards,
> >
> > Vladislav Yaroshchuk
>
>

-- 
Best Regards,

Vladislav Yaroshchuk

[-- Attachment #2: Type: text/html, Size: 12538 bytes --]