From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: qemu-devel@nongnu.org, mlureau@redhat.com,
zhengxiang9@huawei.com, lersek@redhat.com, pbonzini@redhat.com
Subject: Re: [Qemu-devel] [PATCH 2/4] vhost-user: specify and implement VHOST_USER_SET_QUEUE_NUM request
Date: Tue, 16 Jan 2018 15:56:40 +0100 [thread overview]
Message-ID: <b1b27d43-b2d0-3a1e-18cc-92cd2dab95a9@redhat.com> (raw)
In-Reply-To: <20180116050246-mutt-send-email-mst@kernel.org>
On 01/16/2018 04:05 AM, Michael S. Tsirkin wrote:
> On Fri, Jan 12, 2018 at 03:56:56PM +0100, Maxime Coquelin wrote:
>> When the slave cannot add queues dynamically,
>
>
> Could you please clarify the motivation a bit
> Why is it such a big deal to resize the queue array?
> We know no queues are used before all of them are initialized,
> so you don't need to worry about synchronizing with threads
> processing the queues at the same time.
The problem is to know how many queues to wait for being initialized.
We need this information in DPDK to start the "vhost device".
Before the guest driver is initialized, the slave receives from QEMU
VHOST_USER_SET_VRING_CALL and VHOST_USER_SET_VRING_ENABLE for all queues
declared in QEMU. In DPDK, queues metadata is allocated each time a
message targets a new vring index, and vring count is incremented
accordingly.
Currently, it seems to work, because QEMU calls vhost_virtqueue_start()
for all queues, even ones not configured by the guest.
But with patch "[PATCH] vhost: fix corrupting GPA 0 when using
uninitialized queues" from Zheng Xiang, QEMU will not more send
_SET_VRING_ADDR and _SET_VRING_KICK for un-configured queues, but
the backend will wait forever for these messages to start the device.
I pasted below a log of messages received by the slave, so you can make
an idea of the sequence.
>> it needs to know for
>> how many queues to wait to be initialized.
>>
>> This patch introduce new vhost-user protocol feature & request for
>> the master to send the number of queue pairs allocated by the
>> driver.
>
> Assuming we can fix the previous message, I think specifying
> the # of queues would be better.
Thanks,
Maxime
EAL: Detected 4 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:00:1f.6 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 8086:156f net_e1000_em
PMD: Initializing pmd_vhost for net_vhost0
PMD: Creating VHOST-USER backend on numa socket 0
VHOST_CONFIG: vhost-user server: socket created, fd: 13
VHOST_CONFIG: bind to /tmp/vhost-user2
Interactive-mode selected
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=155456,
size=2176, socket=0
Warning! Cannot handle an odd number of ports with the current port
topology. Configuration must be changed to have an even number of ports,
or relaunch application with --port-topology=chained
Configuring Port 0 (socket 0)
Port 0: 56:48:4F:53:54:00
Checking link statuses...
Done
testpmd> VHOST_CONFIG: new vhost user connection is 15
VHOST_CONFIG: new device, handle is 0
VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
VHOST_CONFIG: read message VHOST_USER_GET_PROTOCOL_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_PROTOCOL_FEATURES
VHOST_CONFIG: read message VHOST_USER_GET_QUEUE_NUM
VHOST_CONFIG: read message VHOST_USER_SET_SLAVE_REQ_FD
VHOST_CONFIG: read message VHOST_USER_SET_OWNER
VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:0 file:17
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:1 file:18
VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
VHOST_CONFIG: read message VHOST_USER_GET_PROTOCOL_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_PROTOCOL_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_SLAVE_REQ_FD
VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:2 file:20
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:3 file:21
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 0
PMD: vring0 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 1
PMD: vring1 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 0 to qp idx: 2
PMD: vring2 is disabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 0 to qp idx: 3
PMD: vring3 is disabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 0
PMD: vring0 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 1
PMD: vring1 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 0 to qp idx: 2
PMD: vring2 is disabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 0 to qp idx: 3
PMD: vring3 is disabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE
VHOST_CONFIG: guest memory region 0, size: 0x80000000
guest physical addr: 0x100000000
guest virtual addr: 0x7f9cae200000
host virtual addr: 0x7f633da00000
mmap addr : 0x7f62bda00000
mmap size : 0x100000000
mmap align: 0x200000
mmap off : 0x80000000
VHOST_CONFIG: guest memory region 1, size: 0xa0000
guest physical addr: 0x0
guest virtual addr: 0x7f9c2e200000
host virtual addr: 0x7f64bda00000
mmap addr : 0x7f64bda00000
mmap size : 0x200000
mmap align: 0x200000
mmap off : 0x0
VHOST_CONFIG: guest memory region 2, size: 0x7ff40000
guest physical addr: 0xc0000
guest virtual addr: 0x7f9c2e2c0000
host virtual addr: 0x7f64380c0000
mmap addr : 0x7f6438000000
mmap size : 0x80000000
mmap align: 0x200000
mmap off : 0xc0000
VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
VHOST_CONFIG: vring kick idx:0 file:25
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:0 file:26
VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
VHOST_CONFIG: vring kick idx:1 file:17
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:1 file:27
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 0
PMD: vring0 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 1
PMD: vring1 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
VHOST_CONFIG: vring kick idx:2 file:18
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:2 file:28
VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
VHOST_CONFIG: vring kick idx:3 file:20
VHOST_CONFIG: virtio is now ready for processing.
PMD: New connection established
Port 0: LSC event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:3 file:29
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 0
PMD: vring0 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 1
PMD: vring1 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 2
PMD: vring2 is enabled
Port 0: Queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 3
PMD: vring3 is enabled
Port 0: Queue state event
next prev parent reply other threads:[~2018-01-16 14:57 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-12 14:56 [Qemu-devel] [PATCH 0/4] vhost-user: notify backend with number of queues setup Maxime Coquelin
2018-01-12 14:56 ` [Qemu-devel] [PATCH 1/4] vhost-user: fix multiple queue specification Maxime Coquelin
2018-01-16 3:00 ` Michael S. Tsirkin
2018-01-16 12:35 ` Maxime Coquelin
2018-01-12 14:56 ` [Qemu-devel] [PATCH 2/4] vhost-user: specify and implement VHOST_USER_SET_QUEUE_NUM request Maxime Coquelin
2018-01-16 3:05 ` Michael S. Tsirkin
2018-01-16 14:56 ` Maxime Coquelin [this message]
2018-01-12 14:56 ` [Qemu-devel] [PATCH 3/4] vhost-net: add vhost_net_set_queue_num helper Maxime Coquelin
2018-01-12 14:56 ` [Qemu-devel] [PATCH 4/4] virtio-net: notify backend with number of queue pairs setup Maxime Coquelin
2018-01-16 3:07 ` Michael S. Tsirkin
2018-01-16 14:16 ` Maxime Coquelin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b1b27d43-b2d0-3a1e-18cc-92cd2dab95a9@redhat.com \
--to=maxime.coquelin@redhat.com \
--cc=lersek@redhat.com \
--cc=mlureau@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=zhengxiang9@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).