Re: Question on memory commit during MR finalize()

[Peter Xu <peterx@redhat.com>]

On Mon, Jul 19, 2021 at 08:58:44PM +0000, John Johnson wrote:
> 
> 
> > On Jul 19, 2021, at 12:59 PM, Peter Xu <peterx@redhat.com> wrote:
> > 
> > 
> > Here vfio_user_send_recv() looks tricky to me - it releases the bql within a
> > memory update procedure, and IMHO it needs some serious justification on why it
> > can do so. For example, what if memory layout changed when waiting for the
> > reply?  As it can happen in parallel if without bql, afaict.
> > 
> 
> 
> 	The reason bql is dropped is usually the thread will sleep waiting
> for a reply from the server, and I didn't think it was a good idea to block
> all threads in the meantime.  Most vfio-user requests result from a guest
> action, so just blocking the single CPU thread for the reply is good.

Sleeping with bql is actually ok imho, but indeed if it can take very long then
we should make it async.

> 
> 	The vfio-user code doesn’t depend on the memory layout being stable,
> it’s just sending the layout updates to the server.  Would it be better to
> send memory updates asynchronously, and wait for all the replies in the commit
> callback?

Yes, I believe a lot of similar things are done within qemu, e.g. there can be
a bottom half scheduled so all things will be serialized using bql.

Btw, it's not only about whether vfio-user would survive with memory layout
change, I think the problem is vfio-user now released the bql without qemu core
qemu memory noticing it, while core memory relies on bql to serialize.  That's
why we can get very strange rcu thread crash - it's potentially an outcome of
the race.

Thanks,

-- 
Peter Xu