Re: [PATCH v2 09/11] rust/block: Add read support for block drivers

[Paolo Bonzini <pbonzini@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 8030 bytes --]

Il mer 19 feb 2025, 14:02 Kevin Wolf <kwolf@redhat.com> ha scritto:

> > Likewise, even BochsImage should not need a standard Rust Arc<BdrvChild>.
> > However you need to add your own block::Arc<BdrvChild> and map
> Clone/Drop to
> > bdrv_ref/bdrv_unref.  Then BochsImage can use block::Arc<BdrvChild>; this
> > makes it even clearer that Mapping should not use the Arc<> wrapper,
> because
> > bdrv_ref is GLOBAL_STATE_CODE() and would abort if run from a non-main
> > thread.
>
> It's not BdrvChild that is refcounted on the C side, but
> BlockDriverState. We definitely don't bdrv_ref()/unref() for each
> request on the C side and we shouldn't on the Rust side either. The
> refcount only changes when you modify the graph.
>

I keep confusing BdrvChild and BlockDriverState, sorry.

Talking in general about Rust and not QEMU, if you wanted to be able to
optionally store the mapping, and not always modify the refcount, you'd
probably want an &Arc<T>. Then you only clone it if needed for the cache
(for example if you cached the *content* rather than the mapping, you
wouldn't need to clone).

However, Arc doesn't work here. In order to globally invalidate all cached
mappings, you would need the cache to store something similar to a Weak<T>.
But even that doesn't quite work, because if you passed an &Arc the caller
could clone it as it wished. So using weak clones to do cache invalidation
shows as the hack that it is.

Going back to QEMU, the Mapping needs some smart pointer (something that
implements Deref<Target = bindings;:BdrvChild>), which does implement some
reference counting unlike the C version but, unlike &Arc<BdrvChild>, cannot
be cloned willy nilly. Instead it can be 1) weakly-cloned under the graph
rdlock 2) used from a weak reference but only for the duration of an rdlock
critical section (no way to make it strong for an arbitrary lifetime!) 3)
invalidated under the graph wrlock.

The nice thing here is that, even if you have a good way to invalidate the
cache, that's orthogonal to how you make the Rust API memory safe. Cache
invalidation becomes just a way to quickly free the BdrvChild—the invalid
entry would be detected anyway later when trying to use the weak reference.

More practical/immediate suggestion below...

I'm not entirely sure how your block::Arc<T> is supposed to work. It
> would be tied to one specific type (BlockDriverState), not generic.
> Which probably means that it can't be a separate pointer type, but
> BlockDriverState itself should just implement Clone with bdrv_ref().
>

You're right, I always forget the new BdrvChild world.

> That said, I'm not sure how to include "block graph lock must be taken"
> into
> > the types, yet.  That has to be taken into account too, sooner or later.
> > You probably have a lot of items like this one so it'd be nice to have
> TODO
> > comments as much as you can.
>
> Actually, I'm not aware of that many items.

But yes, there is a TODO
> item for the graph lock.
>
> I think I'll have something like:
>
>     pub struct BdrvChild {
>         child: GraphLock<*mut bindings::BdrvChild>,
>     }
>

Arc<BdrvChild> poses another problem then, in that graph changes would
invalidate the raw pointer even if the Arc is still alive. Something like
the aforementioned smart pointer would prevent the cache from accessing a
dead pointer (at the cost of adding a refcount field to BdrvChild that C
doesn't use).

But for now maybe you can just rename the *mut-wrapping BdrvChild to
BdrvChildRef, get rid of Arc, and store an &BdrvChildRef into Mapping? Then
if you ever decide to go with the refcounting plan you can implement Deref.

> > +) -> std::os::raw::c_int {
> > > +    let s = unsafe { &mut *((*bs).opaque as *mut D) };
> >
> > &mut is not safe here (don't worry, we went through the same thing for
> > devices :)).  You can only get an & unless you go through an UnsafeCell
> (or
> > something that contains one).
>
> Right, we can have multiple requests in flight.
> The fix is easy here: Even though bindgen gives us a *mut, we only want
> a immutable reference.
>

Right.

There is no mutable part in BochsImage, which makes this easy. [...] But if
> we were to introduce a mutable part (I think we will add write

support to it sooner or later), then BqlRefCell or RefCell are
> definitely not right. They would only turn the UB into a safe panic when
> you have more than one request in flight. (Or actually, BqlRefCell
> should already panic with just one request from an iothread, because we
> don't actually hold the BQL.)
>

Yes, I mentioned RefCell because of the iothread case but I agree it also
isn't right. It wouldn't panic when you have more than one request in
flight however, as long as only map() needs to borrow_mut(). If instead you
need slightly more complex locking, for example similar to the vbox driver,
you need CoMutex/CoRwLock bindings.

> > +    let mut offset = offset as u64;
> > > +    let mut bytes = bytes as u64;
> > > +
> > > +    while bytes > 0 {
> > > +        let req = Request::Read { offset, len: bytes };
> > > +        let mapping = match qemu_co_run_future(s.map(&req)) {
> > > +            Ok(mapping) => mapping,
> > > +            Err(e) => return -i32::from(Errno::from(e).0),
> >
> > This is indeed not great, but it's partly so because you're doing a
> > lot (for some definition of "a lot") in the function.  While it would
> > be possible to use a trait, I wrote the API thinking of minimal glue
> > code that only does the C<->Rust conversion.
> >
> > In this case, because you have a lot more code than just a call into
> > the BlockDriver trait, you'd have something like
> >
> > fn bdrv_co_preadv_part(
> >     bs: &dyn BlockDriver,
> >     offset: i64,
> >     bytes: i64,
> >     qiov: &bindings::QEMUIOVector,
> >     mut qiov_offset: usize,
> >     flags: bindings::BdrvRequestFlags) -> io::Result<()>
> >
> > and then a wrapper (e.g. rust_co_preadv_part?) that only does
> >
> >    let s = unsafe { &mut *((*bs).opaque as *mut D) };
> >    let qiov = unsafe { &*qiov };
> >    let result = bdrv_co_preadv_part(s, offset, bytes,
> >          qiov, qiov_offset, flags);
> >    errno::into_negative_errno(result)
> >
> > This by the way has also code size benefits because &dyn, unlike
> > generics, does not need to result in duplicated code.
>
> I don't really like the aesthetics of having two functions on the
> Rust side for each C function, but I guess ugliness is expected in
> bindings...
>

Well, you don't *have* to have two. In this case I suggested two just
because the C and Rust APIs are completely different. But also...

> For now, I'd rather keep into_negative_errno() this way, to keep an
> > eye on other cases where you have an io::Error<()>.  Since Rust rarely
> > has Error objects that aren't part of a Result, it stands to reason
> > that the same is true of QEMU code, but if I'm wrong then it can be
> > changed.
>
> This one is part of a Result, too. But not a result that is directly
> returned in both success and error cases, but where the error leads to
> an early return. That is, an equivalent for the ubiquitous pattern:
>
>     ret = foo();
>     if (ret < 0) {
>         return ret;
>     }
>

... this is just "?" in Rust and it's the C/Rust boundary that complicates
things, because "?" assumes that you return another Result. So the
two-function idea helps because the inner function can just use "?", and
the outer one qemu_api::errno. It let's you use the language more
effectively. &dyn is just an addition on top.

If needed your inner function could return Result<T, Errno> instead of
Result<T, io::Error>, too. That works nicely because "?" would also convert
io::Error into Errno as needed.

>
But if you don't want the two functions, why wouldn't you just do "e =>
..." in the match statement instead of Err(e)?

Paolo

[-- Attachment #2: Type: text/html, Size: 11922 bytes --]