[RFC v5 11/11] docs/zoned-storage: add zoned device documentation

[RFC v5 11/11] docs/zoned-storage: add zoned device documentation

[Sam Li]

Add the documentation about the zoned device support to virtio-blk
emulation.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
---
 docs/devel/zoned-storage.rst           | 68 ++++++++++++++++++++++++++
 docs/system/qemu-block-drivers.rst.inc |  6 +++
 2 files changed, 74 insertions(+)
 create mode 100644 docs/devel/zoned-storage.rst

diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst
new file mode 100644
index 0000000000..e62927dceb
--- /dev/null
+++ b/docs/devel/zoned-storage.rst
@@ -0,0 +1,68 @@
+=============
+zoned-storage
+=============
+
+Zoned Block Devices (ZBDs) devide the LBA space to block regions called zones
+that are larger than the LBA size. It can only allow sequential writes, which
+reduces write amplification in SSD, leading to higher throughput and increased
+capacity. More details about ZBDs can be found at:
+
+https://zonedstorage.io/docs/introduction/zoned-storage
+
+zone emulation
+--------------
+In its current status, the virtio-blk device is not aware of ZBDs but the guest
+sees host-managed drives as regular drive that will runs correctly under the
+most common write workloads.
+
+The zoned device support aims to let guests (virtual machines) access zoned
+storage devices on the host (hypervisor) through a virtio-blk device. This
+involves extending QEMU's block layer and virtio-blk emulation code.
+
+If the host supports zoned block devices, it can set VIRTIO_BLK_F_ZONED. Then
+in the guest side, it appears following situations:
+1) If the guest virtio-blk driver sees the VIRTIO_BLK_F_ZONED bit set, then it
+will assume that the zoned characteristics fields of the config space are valid.
+2) If the guest virtio-blk driver sees a zoned model that is NONE, then it is
+known that is a regular block device.
+3) If the guest virtio-blk driver sees a zoned model that is HM(or HA), then it
+is known that is a zoned block device and probes the other zone fields.
+
+On QEMU sides,
+1) The DEFINE PROP BIT macro must be used to declare that the host supports
+zones.
+2) BlockDrivers can declare zoned device support once known the zoned model
+for the block device is not NONE.
+
+zoned storage APIs
+------------------
+
+Zone emulation part extends the block layer APIs and virtio-blk emulation section
+with the minimum set of zoned commands that are necessary to support zoned
+devices. The commands are - Report Zones, four zone operations and Zone Append
+(developing).
+
+testing
+-------
+
+It can be tested on a null_blk device using qemu-io, qemu-iotests or blkzone(8)
+command in the guest os.
+
+1. For example, the command line for zone report using qemu-io is:
+
+$ path/to/qemu-io --image-opts driver=zoned_host_device,filename=/dev/nullb0 -c
+"zrp offset nr_zones"
+
+To enable zoned device in the guest os, the guest kernel must have the virtio-blk
+driver with ZBDs support. The link to such patches for the kernel is:
+
+https://github.com/dmitry-fomichev/virtblk-zbd
+
+Then, add the following options to the QEMU command line:
+-blockdev node-name=drive0,driver=zoned_host_device,filename=/dev/nullb0
+
+After the guest os booting, use blkzone(8) to test zone operations:
+blkzone report -o offset -c nr_zones /dev/vda
+
+2. We can also use the qemu-iotests in ./tests/qemu-iotests/tests/zoned.sh.
+
diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
index dfe5d2293d..2a761a4b80 100644
--- a/docs/system/qemu-block-drivers.rst.inc
+++ b/docs/system/qemu-block-drivers.rst.inc
@@ -430,6 +430,12 @@ Hard disks
   you may corrupt your host data (use the ``-snapshot`` command
   line option or modify the device permissions accordingly).
 
+Zoned block devices
+  Zoned block devices can passed through to the guest if the emulated storage
+  controller supports zoned storage. Use ``--blockdev zoned_host_device,
+  node-name=drive0,filename=/dev/nullb0`` to pass through ``/dev/nullb0``
+  as ``drive0``.
+
 Windows
 ^^^^^^^
 
-- 
2.37.1

Re: [RFC v5 11/11] docs/zoned-storage: add zoned device documentation

[Stefan Hajnoczi]

On Sun, 31 Jul 2022 at 21:42, Sam Li <faithilikerun@gmail.com> wrote:
>
> Add the documentation about the zoned device support to virtio-blk
> emulation.
>
> Signed-off-by: Sam Li <faithilikerun@gmail.com>
> ---
>  docs/devel/zoned-storage.rst           | 68 ++++++++++++++++++++++++++
>  docs/system/qemu-block-drivers.rst.inc |  6 +++
>  2 files changed, 74 insertions(+)
>  create mode 100644 docs/devel/zoned-storage.rst
>
> diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst
> new file mode 100644
> index 0000000000..e62927dceb
> --- /dev/null
> +++ b/docs/devel/zoned-storage.rst
> @@ -0,0 +1,68 @@
> +=============
> +zoned-storage
> +=============
> +
> +Zoned Block Devices (ZBDs) devide the LBA space to block regions called zones

s/space to block/space into block/

> +that are larger than the LBA size. It can only allow sequential writes, which
> +reduces write amplification in SSD, leading to higher throughput and increased

s/SSD/SSDs/

> +capacity. More details about ZBDs can be found at:
> +
> +https://zonedstorage.io/docs/introduction/zoned-storage
> +
> +zone emulation
> +--------------
> +In its current status, the virtio-blk device is not aware of ZBDs but the guest
> +sees host-managed drives as regular drive that will runs correctly under the
> +most common write workloads.

I don't understand this paragraph. I suggest dropping it. virtio-blk
can be mentioned further down.

> +
> +The zoned device support aims to let guests (virtual machines) access zoned
> +storage devices on the host (hypervisor) through a virtio-blk device. This
> +involves extending QEMU's block layer and virtio-blk emulation code.

This text is specific to your Outreachy project with the goal of
passing through zoned devices via virtio-blk. I suggest writing this
documentation from a broader point of view (emulated storage
controllers and core block layer APIs) and not focussing specifically
on virtio-blk:

1. Block layer APIs for zoned storage
Discuss the QEMU block layer's zoned storage model here. Mention
BlockLimits and how block drivers declare support for zoned storage.

2. Emulating zoned storage controllers
Discuss how emulated storage controllers can use the block layer APIs
when the BlockBackend's BlockLimits model reports a zoned storage
device.

This will help developers who aren't familiar with virtio-blk.
References to virtio-blk make it harder for them to understand how
they should use zoned storage.

> +
> +If the host supports zoned block devices, it can set VIRTIO_BLK_F_ZONED. Then
> +in the guest side, it appears following situations:
> +1) If the guest virtio-blk driver sees the VIRTIO_BLK_F_ZONED bit set, then it
> +will assume that the zoned characteristics fields of the config space are valid.
> +2) If the guest virtio-blk driver sees a zoned model that is NONE, then it is
> +known that is a regular block device.
> +3) If the guest virtio-blk driver sees a zoned model that is HM(or HA), then it
> +is known that is a zoned block device and probes the other zone fields.
> +
> +On QEMU sides,
> +1) The DEFINE PROP BIT macro must be used to declare that the host supports
> +zones.
> +2) BlockDrivers can declare zoned device support once known the zoned model
> +for the block device is not NONE.
> +
> +zoned storage APIs
> +------------------
> +
> +Zone emulation part extends the block layer APIs and virtio-blk emulation section

It's often clearer to describe new features directly instead of the
extensions, changes, or old vs new. The person reading this might not
be familiar with QEMU's block APIs before zoned storage and they don't
care what changed. They just need to know what zoned storage APIs are
available and how to use them.

Here I would say only "The block layer APIs support commands needed
for zoned storage devices, including report zones, four zone
operations, and zone append".

When discussing the zoned storage APIs, avoid mentioning storage
controller emulation as the user of the block API. That is just one
use case. It's possible that zoned storage APIs will be used in other
ways in the future. For example, the qcow2 image format driver could
be optimized for zoned storage devices by laying out and accessing its
metadata on zoned devices using the block APIs. The documentation
should be general enough that someone intending to use the block APIs
differently shouldn't have to know about zoned storage controller
emulation in order to figure out how to use the block APIs for other
purposes.

> +with the minimum set of zoned commands that are necessary to support zoned
> +devices. The commands are - Report Zones, four zone operations and Zone Append
> +(developing).
> +
> +testing
> +-------
> +
> +It can be tested on a null_blk device using qemu-io, qemu-iotests or blkzone(8)
> +command in the guest os.
> +
> +1. For example, the command line for zone report using qemu-io is:
> +
> +$ path/to/qemu-io --image-opts driver=zoned_host_device,filename=/dev/nullb0 -c
> +"zrp offset nr_zones"
> +
> +To enable zoned device in the guest os, the guest kernel must have the virtio-blk
> +driver with ZBDs support. The link to such patches for the kernel is:
> +
> +https://github.com/dmitry-fomichev/virtblk-zbd
> +
> +Then, add the following options to the QEMU command line:
> +-blockdev node-name=drive0,driver=zoned_host_device,filename=/dev/nullb0
> +
> +After the guest os booting, use blkzone(8) to test zone operations:
> +blkzone report -o offset -c nr_zones /dev/vda
> +
> +2. We can also use the qemu-iotests in ./tests/qemu-iotests/tests/zoned.sh.
> +
> diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
> index dfe5d2293d..2a761a4b80 100644
> --- a/docs/system/qemu-block-drivers.rst.inc
> +++ b/docs/system/qemu-block-drivers.rst.inc
> @@ -430,6 +430,12 @@ Hard disks
>    you may corrupt your host data (use the ``-snapshot`` command
>    line option or modify the device permissions accordingly).
>
> +Zoned block devices
> +  Zoned block devices can passed through to the guest if the emulated storage

s/can passed/can be passed/

> +  controller supports zoned storage. Use ``--blockdev zoned_host_device,
> +  node-name=drive0,filename=/dev/nullb0`` to pass through ``/dev/nullb0``
> +  as ``drive0``.
> +
>  Windows
>  ^^^^^^^
>
> --
> 2.37.1
>
>