[RFC] Per-request private data in virtio-block

[RFC] Per-request private data in virtio-block

[Dmitry Fleytman]

Hello QEMU-DEVEL! It's been a while…

I work on a solution for "smart" IO caching on the host side.
The configuration is virtio-block device backed by SPDK with OCF/OpenCAS
on top of remote storage.

To improve decision making for caching, I would like to have an additional
per-request contextual information from the guest side. It might include
information about process issuing an IO request, a file this request it tied
to and so on. In general, I'd like the set of collected information to be
flexible and configurable.

I searched mailing lists and other related sources and surprisingly found no
mentions of the topic of having custom per-request data in virtio rings.
This makes me think that either I'm missing an obvious way of doing this
or the concept itself is severely broken up to the point it's not even
getting discussed. There is also a possibility that I'm just missing a proper
search keywords...

I understand there might be be security implications to be considered.
Also having custom kernel patches to have a flexibility of choosing which
data to collect is probably not a viable solution.

Please share your thoughts.

I would like understand what is the right way of doing what I'm looking for.
If there are new mechanisms to be implemented in virtio or other parts of
the codebase, I'll gladly work on this for the sake of community.

Thank you,
Dmitry

Re: [RFC] Per-request private data in virtio-block

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2636 bytes --]

On Wed, Jul 10, 2024 at 01:08:03PM +0300, Dmitry Fleytman wrote:
> Hello QEMU-DEVEL! It's been a while…
> 
> I work on a solution for "smart" IO caching on the host side.
> The configuration is virtio-block device backed by SPDK with OCF/OpenCAS
> on top of remote storage.
> 
> To improve decision making for caching, I would like to have an additional
> per-request contextual information from the guest side. It might include
> information about process issuing an IO request, a file this request it tied
> to and so on. In general, I'd like the set of collected information to be
> flexible and configurable.
> 
> I searched mailing lists and other related sources and surprisingly found no
> mentions of the topic of having custom per-request data in virtio rings.
> This makes me think that either I'm missing an obvious way of doing this
> or the concept itself is severely broken up to the point it's not even
> getting discussed. There is also a possibility that I'm just missing a proper
> search keywords...
> 
> I understand there might be be security implications to be considered.
> Also having custom kernel patches to have a flexibility of choosing which
> data to collect is probably not a viable solution.
> 
> Please share your thoughts.
> 
> I would like understand what is the right way of doing what I'm looking for.
> If there are new mechanisms to be implemented in virtio or other parts of
> the codebase, I'll gladly work on this for the sake of community.

Hi Dmitry,
Welcome back! The struct virtio_blk_outhdr 32-bit ioprio field is
currently ignored by many device implementations. Is I/O priority
directly related to cache behavior? If yes, your virtio-blk device
implementation could interpret this field. If not, then it's probably
better to add a separate field.

It would be technically possible to add a virtio-blk feature bit for
per-request metadata. Plumbing this new metadata through the stack
requires changes to multiple software components though and might be the
reason why something like this does not exist.

virtio-blk generally sticks to the generic block device model that the
Linux kernel block layer implements. If the Linux block layer doesn't
have the cache metadata concept, then it may require some convincing of
the VIRTIO and Linux communities to add this new concept. Is the concept
of cache metadata (separate from I/O priority in Linux) a thing?

If you can show examples from other storage protocols like NVMe or SCSI,
then that would help in designing a virtio-blk solution too.

Stefan

> 
> Thank you,
> Dmitry
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]