Re: [Qemu-devel] high-level view of packet processing for virtio NIC?

[Dongli Zhang <dongli.zhang@oracle.com>]

Hi Chris,

On 7/24/19 12:18 AM, Chris Friesen wrote:
> Hi,
> 
> I'm looking for information on what the qemu architecture looks like for
> processing virtio network packets in a two-vCPU guest.
> 
> It looks like there's an IO thread doing a decent fraction of the work, separate
> from the vCPU threads--is that correct?  There's no disk involved in this case,
> purely network packet processing.
> 

I suggest use gdb to have a view.

To use gdb, the debug option should be enabled in source code.

# ./configure --target-list=x86_64-softmmu --enable-debug
# make -j8 > /dev/null

# sudo gdb ./x86_64-softmmu/qemu-system-x86_64

(gdb) run -machine pc,accel=kvm -vnc :0 -smp 4 -m 4096M -hda
/path/to/ubuntu1804.qcow2 -device virtio-net-pci,netdev=tapnet,mq=true,vectors=9
-netdev
tap,id=tapnet,ifname=tap0,script=/path/to/qemu-ifup,downscript=no,queues=4,vhost=off

gdb would be able to show which thread is running which function. To have a view
on RX path...

(gdb) b virtio_net_do_receive


Once the break point is hit...

Thread 1 "qemu-system-x86" hit Breakpoint 2, virtio_net_do_receive
(nc=0x555557705d40, buf=0x55555698b2f4 "", size=72) at
/home/zhang/kvm/debug/qemu-4.0.0/hw/net/virtio-net.c:1370
1370	    rcu_read_lock();
(gdb) bt
#0  virtio_net_do_receive (nc=0x555557705d40, buf=0x55555698b2f4 "", size=72) at
/home/zhang/kvm/debug/qemu-4.0.0/hw/net/virtio-net.c:1370
#1  0x00005555558f6934 in virtio_net_receive (nc=0x555557705d40,
buf=0x55555698b2f4 "", size=72) at
/home/zhang/kvm/debug/qemu-4.0.0/hw/net/virtio-net.c:1978
#2  0x0000555555c4208b in nc_sendv_compat (nc=0x555557705d40,
iov=0x7fffffffddb0, iovcnt=1, flags=0) at net/net.c:706
#3  0x0000555555c4214d in qemu_deliver_packet_iov (sender=0x55555698ad00,
flags=0, iov=0x7fffffffddb0, iovcnt=1, opaque=0x555557705d40) at net/net.c:734
#4  0x0000555555c44bae in qemu_net_queue_deliver (queue=0x5555577060d0,
sender=0x55555698ad00, flags=0, data=0x55555698b2f4 "", size=72) at net/queue.c:164
#5  0x0000555555c44cca in qemu_net_queue_send (queue=0x5555577060d0,
sender=0x55555698ad00, flags=0, data=0x55555698b2f4 "", size=72,
sent_cb=0x555555c56829 <tap_send_completed>) at net/queue.c:199
#6  0x0000555555c41ef3 in qemu_send_packet_async_with_flags
(sender=0x55555698ad00, flags=0, buf=0x55555698b2f4 "", size=72,
sent_cb=0x555555c56829 <tap_send_completed>) at net/net.c:660
#7  0x0000555555c41f2b in qemu_send_packet_async (sender=0x55555698ad00,
buf=0x55555698b2f4 "", size=72, sent_cb=0x555555c56829 <tap_send_completed>) at
net/net.c:667
#8  0x0000555555c56938 in tap_send (opaque=0x55555698ad00) at net/tap.c:202
#9  0x0000555555dee376 in aio_dispatch_handlers (ctx=0x5555568770d0) at
util/aio-posix.c:430
#10 0x0000555555dee509 in aio_dispatch (ctx=0x5555568770d0) at util/aio-posix.c:461
#11 0x0000555555de9a20 in aio_ctx_dispatch (source=0x5555568770d0, callback=0x0,
user_data=0x0) at util/async.c:261
#12 0x00007ffff7570197 in g_main_context_dispatch () from
/lib/x86_64-linux-gnu/libglib-2.0.so.0
#13 0x0000555555decdfc in glib_pollfds_poll () at util/main-loop.c:213
#14 0x0000555555dece76 in os_host_main_loop_wait (timeout=198847612) at
util/main-loop.c:236
#15 0x0000555555decf7b in main_loop_wait (nonblocking=0) at util/main-loop.c:512
#16 0x0000555555a0beeb in main_loop () at vl.c:1970
#17 0x0000555555a13230 in main (argc=15, argv=0x7fffffffe3a8,
envp=0x7fffffffe428) at vl.c:4604

Dongli Zhang