Re: [Qemu-devel] [PATCH for-2.4] Revert "vhost-user: Send VHOST_RESET_OWNER on vhost stop"

[Luke Gorrie <luke@snabb.co>]

Hi Michael & all,

On 4 August 2015 at 21:00, Luke Gorrie <luke@snabb.co> wrote:

> Hi Michael,
>
> Sorry I didn't see this mail sooner -
>
> On 30 July 2015 at 10:36, Michael S. Tsirkin <mst@redhat.com> wrote:
>
>> This reverts commit 294ce717e0f212ed0763307f3eab72b4a1bdf4d0.
>>
>> vhost stop happens e.g. when guest unloads the driver,
>> so closing the backend connection is not the right
>> thing to do here.
>>
>> VHOST_RESET_OWNER should happen on vhost_dev_cleanup - it's
>> the counterpart of VHOST_SET_OWNER.
>>
>> Cc: Luke Gorrie <luke@snabb.co>
>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>> ---
>>
>> I think we need this in 2.4 to avoid introducing regressions
>> in the protocol. We'll fix properly in 2.5.
>> Luke, can you comment please?
>>
>
> Interesting. Currently we don't have a test case in Snabb Switch CI for
> guests reloading drivers. We need to add that and let you know how it goes.
>

We have this test case now. Took a year to add, almost to the day, but here
we are :-).

Specifically we have beefed up our CI for Snabb to run some tens of
thousands of end-to-end tests with VMs nightly. This includes testing a
bunch of different QEMU versions with a bunch of different Snabb (host) and
DPDK (guest) versions. It also exercises the transition where the guest
first initializes a Virtio-net device with the kernel driver and then later
hands the device over to the DPDK driver.

The data we have now shows that this test case is only working reliably
with QEMU 2.4.1. My hypothesis is that this QEMU version is sending the
VHOST_RESET_OWNER when the kernel driver shuts down - so that vswitch knows
not to process descriptors - while the other versions are skipping this
(because the patch was not introduced yet, or because the patch was
reverted).

Here is the full thread on Github, becoming more QEMU-oriented as you
scroll down:
https://github.com/snabbco/snabb/issues/976#issuecomment-236838883

So, question for QEMU upstream, is there currently a supported way for the
host vswitch to detect when the vrings are invalid? How should we operate
to be safe from serving garbage DMA requests during guest driver resets and
reboots?

Cheers and sorry for the e-mail latency :-)
-Luke