qcow2-rs v0.1 and rublk-qcow2

qcow2-rs v0.1 and rublk-qcow2

[Ming Lei]

Hello,

qcow2-rs[1] is one pure Rust library for reading/writing qcow2 image, it is
based on rsd's[2] internal qcow2 implementation, but with lots of change, so far:

- supports read/write on data file, backing file and compressed image

- block device like interface, minimized read/write unit is aligned with block
size of image, so that direct io can be supported

- l2 table & refcount block load & store in slice way, and the minimized
slice size is block size, and the maximized size is cluster size

- built over Rust async/await, low level IO handling is abstracted by async
traits, and multiple low level io engines can be supported, so far, verified
on tokio-uring[3], raw linux sync IO syscall and io-uring[4] with smol[5]
runtime

Attributed to excellent async/.await, any IO(include meta IO) is handled in
async way actually, but the programming looks just like writing sync code,
so this library can be well-designed & implemented, and it is easy to add
new features & run further optimization with current code base.

rublk-qcow2[6] wires qcow2-rs, libublk-rs[7], smol(LocalExecutor) and io-uring
together, and provides block device interface for qcow2 image in 500 LoC.

Inside rublk-qcow2 async implementation, io-uring future is mapped to
(waker, result) by using unique cqe.user_data as key via HashMap, this easy way
does work, even though it may slow things a bit, but performance is still not
bad. In simple 'fio/t/io_uring $DEV' test, IOPS of rublk-qcow2 is better than
vdpa-virtio-blk by 20% with same setting(cache.direct=on,aio=io_uring) when
reading from fully allocated image in my test VM.

The initial motivation is for supporting rblk-qcow2, but I can’t find any
Rust qcow2 library with read/write support & simple interfaces and efficient
AIOs support, finally it is evolved into one generic qcow2 library. Many
qcow2 test cases are added. Also one utility is included in this project,
which can dump qcow2 meta, show any meta related statistics of the image,
check image meta integrity & host cluster leak, format qcow2 image,
read & write, ...

Any comments are welcome!



[1] qcow2-rs
https://github.com/ublk-org/qcow2-rs

[2] rsd
https://gitlab.com/hreitz/rsd/-/tree/main/src/node/qcow2?ref_type=heads

[3] tokio-uring
https://docs.rs/tokio-uring

[4] io-uring
https://docs.rs/io-uring

[5] smol
https://docs.rs/smol

[6] rublk-qcow2
https://github.com/ublk-org/rublk

[7] libublk-rs
https://github.com/ublk-org/libublk-rs



Thanks, 
Ming