Re: outw() in virtio_ring_doorbell() in DPDK+virtio consume 40% of the CPU in oprofile

[Stephen Hemminger <stephen-OTpzqLSitTUnbdJkjeBofR2eb7JE58TQ@public.gmane.org>]

On Mon, 16 Dec 2013 15:35:27 -0800
James Yu <ypyu2011-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> (A) The packets I sent are 64-bytes, not big packet. I am not sure GSO will
> help. For bigger packet, it will help.

It will not help with small packets.

> (B) What you do mean "multiple packets per second" ? Do you mean multiple
> queue support to send/receive parallel in multiple cores to speed it up ?
> Is it supported in DPDK 1.3.1r2 ?

With some cases it is possible to get multple packets per send.
This happens if rte_tx_burst is called with more than one packet.

> (C)
> There are two places using dpdk_ring_doorbell() in virtio_user.c,
> eth_tx_burst() and virtio_alloc_rxq() which is called in virtio_recv_buf().
> I looked at them further using "top perf -C 0". It could even occupies 80%
> of the logical core 0 on a CentOS 32-bit VM. Here is the implementation of
> outw() using gcc preprocessing (-E)
> static void outw(unsigned short int value, unsigned short int __port){
>   __asm__ __volatile__ ("outw %w0,%w1": :"a" (value), "Nd" (__port));
> }
> Is outw command a blocking call ?
> Based on this link http://wiki.osdev.org/Inline_Assembly/Examples, I am not
> sure it is blocked/waiting.

Out word causes a VM trap back to hypervisor. Since it is not allowed
as normally by guest, and is used to notify host. Vmware uses memory
in a similar manner as a wakeup.