Re: [Qemu-devel] [PATCH v6 4/4] Add support for net bridge

[Corey Bryant <coreyb@linux.vnet.ibm.com>]

On 12/19/2011 02:36 PM, Anthony Liguori wrote:
> On 12/19/2011 07:11 AM, Corey Bryant wrote:
>> The most common use of -net tap is to connect a tap device to a
>> bridge. This
>> requires the use of a script and running qemu as root in order to
>> allocate a
>> tap device to pass to the script.
>>
>> This model is great for portability and flexibility but it's incredibly
>> difficult to eliminate the need to run qemu as root. The only really
>> viable
>> mechanism is to use tunctl to create a tap device, attach it to a
>> bridge as
>> root, and then hand that tap device to qemu. The problem with this
>> mechanism
>> is that it requires administrator intervention whenever a user wants
>> to create
>> a guest.
>>
>> By essentially writing a helper that implements the most common qemu-ifup
>> script that can be safely given cap_net_admin, we can dramatically
>> simplify
>> things for non-privileged users. We still support existing -net tap
>> options
>> as a mechanism for advanced users and backwards compatibility.
>>
>> Currently, this is very Linux centric but there's really no reason why it
>> couldn't be extended for other Unixes.
>>
>> A typical invocation would be similar to one of the following:
>>
>> qemu linux.img -net bridge -net nic,model=virtio
>>
>> qemu linux.img -net tap,helper=/usr/local/libexec/qemu-bridge-helper
>> -net nic,model=virtio
>>
>> qemu linux.img -netdev bridge,id=hn0
>> -device virtio-net-pci,netdev=hn0,id=nic1
>>
>> qemu linux.img -netdev
>> tap,helper=/usr/local/libexec/qemu-bridge-helper,id=hn0
>> -device virtio-net-pci,netdev=hn0,id=nic1
>>
>> The default bridge that we attach to is br0. The thinking is that a
>> distro
>> could preconfigure such an interface to allow out-of-the-box bridged
>> networking.
>>
>> Alternatively, if a user wants to use a different bridge, a typical
>> invocation
>> would be simliar to one of the following:
>>
>> qemu linux.img -net bridge,br=qemubr0 -net nic,model=virtio
>>
>> qemu linux.img -net
>> tap,helper=/usr/local/libexec/qemu-bridge-helper,br=qemubr0
>> -net nic,model=virtio
>>
>> qemu linux.img -netdev bridge,br=qemubr0,id=hn0
>> -device virtio-net-pci,netdev=hn0,id=nic1
>>
>> qemu linux.img -netdev
>> tap,helper=/usr/local/libexec/qemu-bridge-helper,br=qemubr0,id=hn0
>> -device virtio-net-pci,netdev=hn0,id=nic1
>>
>> Signed-off-by: Anthony Liguori<aliguori@us.ibm.com>
>> Signed-off-by: Richa Marwaha<rmarwah@linux.vnet.ibm.com>
>> Signed-off-by: Corey Bryant<coreyb@linux.vnet.ibm.com>
>> ---
>> configure | 2 +
>> net.c | 29 ++++++++-
>> net.h | 3 +
>> net/tap.c | 187 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>> net/tap.h | 3 +
>> qemu-options.hx | 74 ++++++++++++++++++----
>> 6 files changed, 281 insertions(+), 17 deletions(-)
>>
>> diff --git a/configure b/configure
>> index 6ed4196..4839694 100755
>> --- a/configure
>> +++ b/configure
>> @@ -2905,6 +2905,8 @@ echo "sysconfdir=$sysconfdir">> $config_host_mak
>> echo "docdir=$docdir">> $config_host_mak
>> echo "confdir=$confdir">> $config_host_mak
>> echo "libexecdir=\${prefix}/libexec">> $config_host_mak
>> +echo "CONFIG_QEMU_SHAREDIR=\"$prefix$datasuffix\"">> $config_host_mak
>> +echo "CONFIG_QEMU_HELPERDIR=\"$prefix/libexec\"">> $config_host_mak
>>
>> case "$cpu" in
>> i386|x86_64|alpha|arm|cris|hppa|ia64|lm32|m68k|microblaze|mips|mips64|ppc|ppc64|s390|s390x|sparc|sparc64|unicore32)
>>
>> diff --git a/net.c b/net.c
>> index f7bebf8..9296224 100644
>> --- a/net.c
>> +++ b/net.c
>> @@ -952,6 +952,14 @@ static const struct {
>> .type = QEMU_OPT_STRING,
>> .help = "script to shut down the interface",
>> }, {
>> + .name = "br",
>> + .type = QEMU_OPT_STRING,
>> + .help = "bridge name",
>> + }, {
>
> I don't think passing br= makes a whole of sense for -net tap. I think
> it would make more sense to make sure that helper could take a shell
> string so you could do:
>
> -netdev tap,helper="/usr/libexec/qemu-bridge-helper --br=br0"
>
> Regards,
>
> Anthony Liguori
>

Ok but do you think the -net bridge options should remain as-is?  It 
seems like execution of the helper should be consistent.  Here are the 
current options for -net bridge:

-net bridge,helper=/usr/local/libexec/qemu-bridge-helper,br=br0

-- 
Regards,
Corey

>> + .name = "helper",
>> + .type = QEMU_OPT_STRING,
>> + .help = "command to execute to configure bridge",
>> + }, {
>> .name = "sndbuf",
>> .type = QEMU_OPT_SIZE,
>> .help = "send buffer limit"
>> @@ -1049,6 +1057,23 @@ static const struct {
>> { /* end of list */ }
>> },
>> },
>> + [NET_CLIENT_TYPE_BRIDGE] = {
>> + .type = "bridge",
>> + .init = net_init_bridge,
>> + .desc = {
>> + NET_COMMON_PARAMS_DESC,
>> + {
>> + .name = "br",
>> + .type = QEMU_OPT_STRING,
>> + .help = "bridge name",
>> + }, {
>> + .name = "helper",
>> + .type = QEMU_OPT_STRING,
>> + .help = "command to execute to configure bridge",
>> + },
>> + { /* end of list */ }
>> + },
>> + },
>> };
>>
>> int net_client_init(Monitor *mon, QemuOpts *opts, int is_netdev)
>> @@ -1071,7 +1096,8 @@ int net_client_init(Monitor *mon, QemuOpts
>> *opts, int is_netdev)
>> #ifdef CONFIG_VDE
>> strcmp(type, "vde") != 0&&
>> #endif
>> - strcmp(type, "socket") != 0) {
>> + strcmp(type, "socket") != 0&&
>> + strcmp(type, "bridge") != 0) {
>> qerror_report(QERR_INVALID_PARAMETER_VALUE, "type",
>> "a netdev backend type");
>> return -1;
>> @@ -1141,6 +1167,7 @@ static int net_host_check_device(const char
>> *device)
>> #ifdef CONFIG_VDE
>> ,"vde"
>> #endif
>> + , "bridge"
>> };
>> for (i = 0; i< sizeof(valid_param_list) / sizeof(char *); i++) {
>> if (!strncmp(valid_param_list[i], device,
>> diff --git a/net.h b/net.h
>> index c6b4190..0fd7e23 100644
>> --- a/net.h
>> +++ b/net.h
>> @@ -36,6 +36,7 @@ typedef enum {
>> NET_CLIENT_TYPE_SOCKET,
>> NET_CLIENT_TYPE_VDE,
>> NET_CLIENT_TYPE_DUMP,
>> + NET_CLIENT_TYPE_BRIDGE,
>>
>> NET_CLIENT_TYPE_MAX
>> } net_client_type;
>> @@ -173,6 +174,8 @@ int do_netdev_del(Monitor *mon, const QDict
>> *qdict, QObject **ret_data);
>>
>> #define DEFAULT_NETWORK_SCRIPT "/etc/qemu-ifup"
>> #define DEFAULT_NETWORK_DOWN_SCRIPT "/etc/qemu-ifdown"
>> +#define DEFAULT_BRIDGE_HELPER CONFIG_QEMU_HELPERDIR
>> "/qemu-bridge-helper"
>> +#define DEFAULT_BRIDGE_INTERFACE "br0"
>>
>> void qdev_set_nic_properties(DeviceState *dev, NICInfo *nd);
>>
>> diff --git a/net/tap.c b/net/tap.c
>> index 6c27a94..b2b82a1 100644
>> --- a/net/tap.c
>> +++ b/net/tap.c
>> @@ -382,6 +382,143 @@ static int launch_script(const char
>> *setup_script, const char *ifname, int fd)
>> return -1;
>> }
>>
>> +static int recv_fd(int c)
>> +{
>> + int fd;
>> + uint8_t msgbuf[CMSG_SPACE(sizeof(fd))];
>> + struct msghdr msg = {
>> + .msg_control = msgbuf,
>> + .msg_controllen = sizeof(msgbuf),
>> + };
>> + struct cmsghdr *cmsg;
>> + struct iovec iov;
>> + uint8_t req[1];
>> + ssize_t len;
>> +
>> + cmsg = CMSG_FIRSTHDR(&msg);
>> + cmsg->cmsg_level = SOL_SOCKET;
>> + cmsg->cmsg_type = SCM_RIGHTS;
>> + cmsg->cmsg_len = CMSG_LEN(sizeof(fd));
>> + msg.msg_controllen = cmsg->cmsg_len;
>> +
>> + iov.iov_base = req;
>> + iov.iov_len = sizeof(req);
>> +
>> + msg.msg_iov =&iov;
>> + msg.msg_iovlen = 1;
>> +
>> + len = recvmsg(c,&msg, 0);
>> + if (len> 0) {
>> + memcpy(&fd, CMSG_DATA(cmsg), sizeof(fd));
>> + return fd;
>> + }
>> +
>> + return len;
>> +}
>> +
>> +static int net_bridge_run_helper(const char *helper, const char *bridge)
>> +{
>> + sigset_t oldmask, mask;
>> + int pid, status;
>> + char *args[5];
>> + char **parg;
>> + int sv[2];
>> +
>> + sigemptyset(&mask);
>> + sigaddset(&mask, SIGCHLD);
>> + sigprocmask(SIG_BLOCK,&mask,&oldmask);
>> +
>> + if (socketpair(PF_UNIX, SOCK_STREAM, 0, sv) == -1) {
>> + return -1;
>> + }
>> +
>> + /* try to launch bridge helper */
>> + pid = fork();
>> + if (pid == 0) {
>> + int open_max = sysconf(_SC_OPEN_MAX), i;
>> + char buf[32];
>> +
>> + snprintf(buf, sizeof(buf), "%d", sv[1]);
>> +
>> + for (i = 0; i< open_max; i++) {
>> + if (i != STDIN_FILENO&&
>> + i != STDOUT_FILENO&&
>> + i != STDERR_FILENO&&
>> + i != sv[1]) {
>> + close(i);
>> + }
>> + }
>> + parg = args;
>> + *parg++ = (char *)helper;
>> + *parg++ = (char *)"--use-vnet";
>> + *parg++ = (char *)bridge;
>> + *parg++ = buf;
>> + *parg++ = NULL;
>> + execv(helper, args);
>> + _exit(1);
>> + } else if (pid> 0) {
>> + int fd;
>> +
>> + close(sv[1]);
>> +
>> + do {
>> + fd = recv_fd(sv[0]);
>> + } while (fd == -1&& errno == EINTR);
>> +
>> + close(sv[0]);
>> +
>> + while (waitpid(pid,&status, 0) != pid) {
>> + /* loop */
>> + }
>> + sigprocmask(SIG_SETMASK,&oldmask, NULL);
>> + if (fd< 0) {
>> + fprintf(stderr, "failed to recv file descriptor\n");
>> + return -1;
>> + }
>> +
>> + if (WIFEXITED(status)&& WEXITSTATUS(status) == 0) {
>> + return fd;
>> + }
>> + }
>> + fprintf(stderr, "failed to launch bridge helper\n");
>> + return -1;
>> +}
>> +
>> +int net_init_bridge(QemuOpts *opts, Monitor *mon, const char *name,
>> + VLANState *vlan)
>> +{
>> + TAPState *s;
>> + int fd, vnet_hdr;
>> +
>> + if (!qemu_opt_get(opts, "br")) {
>> + qemu_opt_set(opts, "br", DEFAULT_BRIDGE_INTERFACE);
>> + }
>> + if (!qemu_opt_get(opts, "helper")) {
>> + qemu_opt_set(opts, "helper", DEFAULT_BRIDGE_HELPER);
>> + }
>> +
>> + fd = net_bridge_run_helper(qemu_opt_get(opts, "helper"),
>> + qemu_opt_get(opts, "br"));
>> + if (fd == -1) {
>> + return -1;
>> + }
>> +
>> + fcntl(fd, F_SETFL, O_NONBLOCK);
>> +
>> + vnet_hdr = tap_probe_vnet_hdr(fd);
>> +
>> + s = net_tap_fd_init(vlan, "bridge", name, fd, vnet_hdr);
>> + if (!s) {
>> + close(fd);
>> + return -1;
>> + }
>> +
>> + snprintf(s->nc.info_str, sizeof(s->nc.info_str),
>> + "br=%s", qemu_opt_get(opts, "br"));
>> +
>> + return 0;
>> +}
>> +
>> static int net_tap_init(QemuOpts *opts, int *vnet_hdr)
>> {
>> int fd, vnet_hdr_required;
>> @@ -422,13 +559,17 @@ int net_init_tap(QemuOpts *opts, Monitor *mon,
>> const char *name, VLANState *vlan
>> {
>> TAPState *s;
>> int fd, vnet_hdr = 0;
>> + const char *model;
>>
>> if (qemu_opt_get(opts, "fd")) {
>> if (qemu_opt_get(opts, "ifname") ||
>> qemu_opt_get(opts, "script") ||
>> qemu_opt_get(opts, "downscript") ||
>> - qemu_opt_get(opts, "vnet_hdr")) {
>> - error_report("ifname=, script=, downscript= and vnet_hdr= is invalid
>> with fd=");
>> + qemu_opt_get(opts, "vnet_hdr") ||
>> + qemu_opt_get(opts, "br") ||
>> + qemu_opt_get(opts, "helper")) {
>> + error_report("ifname=, script=, downscript=, vnet_hdr=, "
>> + "br= and helper= are invalid with fd=");
>> return -1;
>> }
>>
>> @@ -440,7 +581,41 @@ int net_init_tap(QemuOpts *opts, Monitor *mon,
>> const char *name, VLANState *vlan
>> fcntl(fd, F_SETFL, O_NONBLOCK);
>>
>> vnet_hdr = tap_probe_vnet_hdr(fd);
>> +
>> + model = "tap";
>> +
>> + } else if (qemu_opt_get(opts, "helper")) {
>> + if (qemu_opt_get(opts, "ifname") ||
>> + qemu_opt_get(opts, "script") ||
>> + qemu_opt_get(opts, "downscript") ||
>> + qemu_opt_get(opts, "vnet_hdr")) {
>> + error_report("ifname=, script=, downscript=, and vnet_hdr= "
>> + "are invalid with helper=");
>> + return -1;
>> + }
>> +
>> + if (!qemu_opt_get(opts, "br")) {
>> + qemu_opt_set(opts, "br", DEFAULT_BRIDGE_INTERFACE);
>> + }
>> +
>> + fd = net_bridge_run_helper(qemu_opt_get(opts, "helper"),
>> + qemu_opt_get(opts, "br"));
>> + if (fd == -1) {
>> + return -1;
>> + }
>> +
>> + fcntl(fd, F_SETFL, O_NONBLOCK);
>> +
>> + vnet_hdr = tap_probe_vnet_hdr(fd);
>> +
>> + model = "bridge";
>> +
>> } else {
>> + if (qemu_opt_get(opts, "br")) {
>> + error_report("br= is invalid with script=");
>> + return -1;
>> + }
>> +
>> if (!qemu_opt_get(opts, "script")) {
>> qemu_opt_set(opts, "script", DEFAULT_NETWORK_SCRIPT);
>> }
>> @@ -453,9 +628,11 @@ int net_init_tap(QemuOpts *opts, Monitor *mon,
>> const char *name, VLANState *vlan
>> if (fd == -1) {
>> return -1;
>> }
>> +
>> + model = "tap";
>> }
>>
>> - s = net_tap_fd_init(vlan, "tap", name, fd, vnet_hdr);
>> + s = net_tap_fd_init(vlan, model, name, fd, vnet_hdr);
>> if (!s) {
>> close(fd);
>> return -1;
>> @@ -467,6 +644,10 @@ int net_init_tap(QemuOpts *opts, Monitor *mon,
>> const char *name, VLANState *vlan
>>
>> if (qemu_opt_get(opts, "fd")) {
>> snprintf(s->nc.info_str, sizeof(s->nc.info_str), "fd=%d", fd);
>> + } else if (qemu_opt_get(opts, "helper")) {
>> + snprintf(s->nc.info_str, sizeof(s->nc.info_str),
>> + "helper=%s,br=%s", qemu_opt_get(opts, "helper"),
>> + qemu_opt_get(opts, "br"));
>> } else {
>> const char *ifname, *script, *downscript;
>>
>> diff --git a/net/tap.h b/net/tap.h
>> index e44bd2b..56c591f 100644
>> --- a/net/tap.h
>> +++ b/net/tap.h
>> @@ -57,4 +57,7 @@ int tap_get_fd(VLANClientState *vc);
>> struct vhost_net;
>> struct vhost_net *tap_get_vhost_net(VLANClientState *vc);
>>
>> +int net_init_bridge(QemuOpts *opts, Monitor *mon, const char *name,
>> + VLANState *vlan);
>> +
>> #endif /* QEMU_NET_TAP_H */
>> diff --git a/qemu-options.hx b/qemu-options.hx
>> index 087a3b9..4f2385d 100644
>> --- a/qemu-options.hx
>> +++ b/qemu-options.hx
>> @@ -1205,11 +1205,14 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
>> "-net tap[,vlan=n][,name=str],ifname=name\n"
>> " connect the host TAP network interface to VLAN 'n'\n"
>> #else
>> - "-net
>> tap[,vlan=n][,name=str][,fd=h][,ifname=name][,script=file][,downscript=dfile][,sndbuf=nbytes][,vnet_hdr=on|off][,vhost=on|off][,vhostfd=h][,vhostforce=on|off]\n"
>>
>> - " connect the host TAP network interface to VLAN 'n' and use the\n"
>> - " network scripts 'file' (default=" DEFAULT_NETWORK_SCRIPT ")\n"
>> - " and 'dfile' (default=" DEFAULT_NETWORK_DOWN_SCRIPT ")\n"
>> + "-net
>> tap[,vlan=n][,name=str][,fd=h][,ifname=name][,script=file][,downscript=dfile][,br=bridge][,helper=helper][,sndbuf=nbytes][,vnet_hdr=on|off][,vhost=on|off][,vhostfd=h][,vhostforce=on|off]\n"
>>
>> + " connect the host TAP network interface to VLAN 'n' \n"
>> + " use network scripts 'file' (default=" DEFAULT_NETWORK_SCRIPT ")\n"
>> + " to configure it and 'dfile' (default=" DEFAULT_NETWORK_DOWN_SCRIPT
>> ")\n"
>> + " to deconfigure it\n"
>> " use '[down]script=no' to disable script execution\n"
>> + " use network helper 'helper' (default=" DEFAULT_BRIDGE_HELPER ")
>> and\n"
>> + " bridge 'br' (default=" DEFAULT_BRIDGE_INTERFACE ") to configure it\n"
>> " use 'fd=h' to connect to an already opened TAP interface\n"
>> " use 'sndbuf=nbytes' to limit the size of the send buffer (the\n"
>> " default is disabled 'sndbuf=0' to enable flow control set
>> 'sndbuf=1048576')\n"
>> @@ -1219,6 +1222,10 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
>> " (only has effect for virtio guests which use MSIX)\n"
>> " use vhostforce=on to force vhost on for non-MSIX virtio guests\n"
>> " use 'vhostfd=h' to connect to an already opened vhost net device\n"
>> + "-net bridge[,vlan=n][,name=str][,br=bridge][,helper=helper]\n"
>> + " connects a host TAP network interface to a host bridge device 'br'\n"
>> + " (default=" DEFAULT_BRIDGE_INTERFACE ") using the program 'helper'\n"
>> + " (default=" DEFAULT_BRIDGE_HELPER ")\n"
>> #endif
>> "-net
>> socket[,vlan=n][,name=str][,fd=h][,listen=[host]:port][,connect=host:port]\n"
>>
>> " connect the vlan 'n' to another VLAN using a socket connection\n"
>> @@ -1242,6 +1249,7 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev,
>> "user|"
>> #endif
>> "tap|"
>> + "bridge|"
>> #ifdef CONFIG_VDE
>> "vde|"
>> #endif
>> @@ -1378,26 +1386,66 @@ processed and applied to -net user. Mixing
>> them with the new configuration
>> syntax gives undefined results. Their use for new applications is
>> discouraged
>> as they will be removed from future versions.
>>
>> -@item -net
>> tap[,vlan=@var{n}][,name=@var{name}][,fd=@var{h}][,ifname=@var{name}]
>> [,script=@var{file}][,downscript=@var{dfile}]
>> -Connect the host TAP network interface @var{name} to VLAN @var{n}, use
>> -the network script @var{file} to configure it and the network script
>> +@item -net
>> tap[,vlan=@var{n}][,name=@var{name}][,fd=@var{h}][,ifname=@var{name}][,script=@var{file}][,downscript=@var{dfile}][,br=@var{bridge}][,helper=@var{helper}]
>>
>> +Connect the host TAP network interface @var{name} to VLAN @var{n}.
>> +
>> +Use the network script @var{file} to configure it and the network script
>> @var{dfile} to deconfigure it. If @var{name} is not provided, the OS
>> -automatically provides one. @option{fd}=@var{h} can be used to specify
>> -the handle of an already opened host TAP interface. The default network
>> -configure script is @file{/etc/qemu-ifup} and the default network
>> -deconfigure script is @file{/etc/qemu-ifdown}. Use @option{script=no}
>> -or @option{downscript=no} to disable script execution. Example:
>> +automatically provides one. The default network configure script is
>> +@file{/etc/qemu-ifup} and the default network deconfigure script is
>> +@file{/etc/qemu-ifdown}. Use @option{script=no} or
>> @option{downscript=no}
>> +to disable script execution.
>> +
>> +If running QEMU as an unprivileged user, use the network helper
>> +@var{helper} to configure the TAP interface. The default network
>> +helper executable is @file{/usr/local/libexec/qemu-bridge-helper}
>> +and the default bridge device is @file{br0}.
>> +
>> +@option{fd}=@var{h} can be used to specify the handle of an already
>> +opened host TAP interface.
>> +
>> +Examples:
>>
>> @example
>> +#launch a QEMU instance with the default network script
>> qemu linux.img -net nic -net tap
>> @end example
>>
>> -More complicated example (two NICs, each one connected to a TAP device)
>> @example
>> +#launch a QEMU instance with two NICs, each one connected
>> +#to a TAP device
>> qemu linux.img -net nic,vlan=0 -net tap,vlan=0,ifname=tap0 \
>> -net nic,vlan=1 -net tap,vlan=1,ifname=tap1
>> @end example
>>
>> +@example
>> +#launch a QEMU instance with the default network helper to
>> +#connect a TAP device to bridge br0
>> +qemu linux.img -net nic -net
>> tap,helper=/usr/local/libexec/qemu-bridge-helper
>> +@end example
>> +
>> +@item -net
>> bridge[,vlan=@var{n}][,name=@var{name}][,br=@var{bridge}][,helper=@var{helper}]
>>
>> +Connect a host TAP network interface to a host bridge device.
>> +
>> +Use the network helper @var{helper} to configure the TAP interface and
>> +attach it to the bridge. The default network helper executable is
>> +@file{/usr/local/libexec/qemu-bridge-helper} and the default bridge
>> +device is @file{br0}.
>> +
>> +Examples:
>> +
>> +@example
>> +#launch a QEMU instance with the default network helper to
>> +#connect a TAP device to bridge br0
>> +qemu linux.img -net bridge -net nic,model=virtio
>> +@end example
>> +
>> +@example
>> +#launch a QEMU instance with the default network helper to
>> +#connect a TAP device to bridge qemubr0
>> +qemu linux.img -net bridge,br=qemubr0 -net nic,model=virtio
>> +@end example
>> +
>> @item -net socket[,vlan=@var{n}][,name=@var{name}][,fd=@var{h}]
>> [,listen=[@var{host}]:@var{port}][,connect=@var{host}:@var{port}]
>>
>> Connect the VLAN @var{n} to a remote VLAN in another QEMU virtual
>