vhost-net guest memory usage

vhost-net guest memory usage

[Arjan Koers]

When running my guests with 'vhost=on', a lot less guest memory is
available (about 18.5 MB less for each networking device).

I searched the mailing list archive and found a comment by Michael S.
Tsirkin:
"with vhost, virtio does currently consume a bit more memory than with
userspace backend"

Is 18.5 MB per device the expected amount, and can I change some
setting to reduce it?


Here's the 'free' output of a guest with 7 tap devices (qemu-kvm 0.13.0,
linux 2.6.36.2 host and guest):

Started the guest with 'vhost=off':
	             total       used       free     shared    buffers     cached
	Mem:        253564      39260     214304          0       1732      15124
	-/+ buffers/cache:      22404     231160
	Swap:       409620          0     409620

Same guest, but with 'vhost=on':
	             total       used       free     shared    buffers     cached
	Mem:        253564     169092      84472          0       1744      15120
	-/+ buffers/cache:     152228     101336
	Swap:       409620          0     409620

The command that started the guest:
/opt/kvm/bin/qemu-system-x86_64 \
  -nographic \
  -daemonize \
  -cpu host \
  -M pc-0.13 \
  -m 256 \
  -mem-path /hugepages \
  -smp 2 \
  -name gw \
  -uuid d9d6332d-10ba-4cfa-925b-4e4d14f7cd86 \
  -boot c \
  -drive file=/opt/kvm/var/img/gw.raw,if=virtio,index=0,boot=on,cache=none \
  -device virtio-net-pci,netdev=virtio.0,mac=fe:60:20:3a:8d:37 \
  -netdev tap,id=virtio.0,ifname=gw-lan,script=no,downscript=no,vhost=on \
  -device virtio-net-pci,netdev=virtio.1,mac=06:c8:4d:85:47:45 \
  -netdev tap,id=virtio.1,ifname=gw-dmz,script=no,downscript=no,vhost=on \
  -device virtio-net-pci,netdev=virtio.2,mac=e2:57:e2:6b:67:7c \
  -netdev tap,id=virtio.2,ifname=gw-inet,script=no,downscript=no,vhost=on \
  -device virtio-net-pci,netdev=virtio.3,mac=72:47:f5:4d:59:01 \
  -netdev tap,id=virtio.3,ifname=gw-lanv,script=no,downscript=no,vhost=on \
  -device virtio-net-pci,netdev=virtio.4,mac=02:de:c4:7b:71:6e \
  -netdev tap,id=virtio.4,ifname=gw-dmzv,script=no,downscript=no,vhost=on \
  -device virtio-net-pci,netdev=virtio.5,mac=fe:6a:ac:4e:16:f3 \
  -netdev tap,id=virtio.5,ifname=gw-lanvpn,script=no,downscript=no,vhost=on \
  -device virtio-net-pci,netdev=virtio.6,mac=7e:33:75:7a:ec:cd \
  -netdev tap,id=virtio.6,ifname=gw-dmzvpn,script=no,downscript=no,vhost=on \
  -serial unix:/opt/kvm/var/run/gw-serial.socket,server,nowait \
  -monitor unix:/opt/kvm/var/run/gw-monitor.socket,server,nowait \
  -parallel none \
  -vga cirrus \
  -runas kvm-gw