Re: [PATCH 14/15] fuse: Implement multi-threading

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3136 bytes --]

On Tue, Apr 01, 2025 at 03:36:40PM -0500, Eric Blake wrote:
> On Thu, Mar 27, 2025 at 11:55:57AM -0400, Stefan Hajnoczi wrote:
> > On Tue, Mar 25, 2025 at 05:06:54PM +0100, Hanna Czenczek wrote:
> > > FUSE allows creating multiple request queues by "cloning" /dev/fuse FDs
> > > (via open("/dev/fuse") + ioctl(FUSE_DEV_IOC_CLONE)).
> > > 
> > > We can use this to implement multi-threading.
> > > 
> > > Note that the interface presented here differs from the multi-queue
> > > interface of virtio-blk: The latter maps virtqueues to iothreads, which
> > > allows processing multiple virtqueues in a single iothread.  The
> > > equivalent (processing multiple FDs in a single iothread) would not make
> > > sense for FUSE because those FDs are used in a round-robin fashion by
> > > the FUSE kernel driver.  Putting two of them into a single iothread will
> > > just create a bottleneck.
> > 
> > This text might be outdated. virtio-blk's new iothread-vq-mapping
> > parameter provides the "array of iothreads" mentioned below and a way to
> > assign virtqueues to those IOThreads.
> > 
> 
> > > +++ b/qapi/block-export.json
> > > @@ -179,12 +179,18 @@
> > >  #     mount the export with allow_other, and if that fails, try again
> > >  #     without.  (since 6.1; default: auto)
> > >  #
> > > +# @iothreads: Enables multi-threading: Handle requests in each of the
> > > +#     given iothreads (instead of the block device's iothread, or the
> > > +#     export's "main" iothread).  For this, the FUSE FD is duplicated so
> > > +#     there is one FD per iothread.  (since 10.1)
> > 
> > This option isn't FUSE-specific but FUSE is the first export type to
> > support it. Please add it to BlockExportOptions instead and refuse
> > export creation when the export type only supports 1 IOThread.
> > 
> > Eric: Are you interested in implementing support for multiple IOThreads
> > in the NBD export? I remember some time ago we talked about NBD
> > multi-conn support, although maybe that was for the client rather than
> > the server.
> 
> The NBD server already supports clients that make requests through
> multiple TCP sockets, but right now, that server is not taking
> advantage of iothreads to spread the TCP load.
> 
> And yes, I am in the middle of working on adding client NBD multi-conn
> support (reviving Rich Jones' preliminary patches on what it would
> take to have qemu open parallel TCP sockets to a supporting NBD
> server), which also will use a round-robin approach (but here, the
> round-robin is something we would code up in qemu, rather than the
> behavior handed to us by the FUSE kernel layer).  Pinning specific
> iothreads to a specific TCP socket may or may not make sense, but I
> definitely want to have support for handing a pool of iothreads to an
> NBD client that will be using multi-conn.

Here I'm asking Hanna to make the "iothreads" export parameter generic
for all export types (not just for FUSE). Do you think that the NBD
export will be able to use the generic parameter or would you prefer an
NBD-specific export parameter?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]