Re: [Qemu-devel] qcow2: Can create qcow2 image format on rbd server

[Jun Li <junmuzi@gmail.com>]

On Tue, 12/09 09:49, Stefan Hajnoczi wrote:
> On Tue, Dec 9, 2014 at 3:52 AM, Josh Durgin <josh.durgin@inktank.com> wrote:
> > On 12/08/2014 05:58 AM, Jun Li wrote:
> >>
> >> On Fri, 12/05 18:01, Max Reitz wrote:
> >>>
> >>> On 2014-12-05 at 16:32, Jun Li wrote:
> >>>>
> >>>> Currently, qemu-img can not create qcow2 image format on rbd server.
> >>>> Analysis
> >>>> the code as followings:
> >>>> when create qcow2 format image:
> >>>> qcow2_create2
> >>>>    bdrv_create_file(filename, opts, &local_err);  --> Here will create a
> >>>> 0 size
> >>>>    file(e.g: file1) on rbd server.
> >>>>    ...
> >>>>    ret = bdrv_pwrite(bs, 0, header, cluster_size); --> So here can not
> >>>> write
> >>>>    qcow2 header into file1 due to the file1's length is 0. Seems
> >>>>    qemu_rbd_aio_writev can not write beyond EOF.
> >>>>    ...
> >>>>
> >>>> As above analysis, there are two methods to solve the above bz as
> >>>> followings:
> >>>> 1, When create file1, just create a fixed-size file1 on rbd server(not 0
> >>>> size).
> >>>
> >>>
> >>> Should be possible by using -o preallocation=falloc or -o
> >>> preallocation=full.
> >>
> >>
> >> Although "-o preallocation=falloc or -o preallocation=full" can create a
> >> qcow2
> >> format image successfully, but when perform "qemu-img resize file.qcow2
> >> +500M", then use the extend 500M disk image still hit the same issue(as
> >> rbd
> >> block driver does not support growable file).
> >
> >
> > Why not use 'rbd resize' and raw images instead?
> > rbd already supports snapshots, cloning, thin provisioning, and
> > differential backup natively, so putting qcow2 on top tends to just add
> > overhead.
> 
> In general, I don't expect many people to use qcow2 on rbd either.
> 
> However, qcow2 does work on top of iSCSI, LVM, or host block devices
> and some users actually use this feature (even though there is LVM
> thin provisioning, for example).  So the failure on rbd is unexpected
> and inconsistent.
> 
> The way it works on other protocols is that the user must first create
> an adequately sized volume before running qemu-img create.  With rbd
> this doesn't work because we truncate the volume to 0 bytes during
> create.
> 
> Either we need to fix this (without losing the ability to qemu-img
> create -f raw rbd:... 10G) or we should have a clear error message.

what you do mean by "fix this"? Realize a growable rbd block driver? If not,
seems we should have a clear error message currently.

I have attempted to realize a asynchronous rbd_resize at qemu level(create a
child-thread to perform rbd_resize in qemu_rbd_aio_writev), but it failed due
to the librbd. Gdb debug info just like followings:
(gdb) bt
#0  librbd::resize (ictx=0x48fffe948fe80574, size=120095921237475840,
				prog_ctx=...) at librbd/internal.cc:1492
#1  0x00007ffff7478bb2 in rbd_resize (image=<optimized out>, size=<optimized
				out>) at librbd/librbd.cc:726
#2  0x00005555555cfa9c in qemu_rbd_truncate (bs=0x7ffff7f9ca00,
				offset=120095921237475840) at block/rbd.c:798
#3  0x00005555555cf797 in thread_info (t_parameters=0x7ffff7f9c9d0) at
block/rbd.c:706
#4  0x00007ffff1c5bee5 in start_thread (arg=0x7fffd5ffb700) at
pthread_create.c:309
#5  0x00007ffff198ab8d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Based on above, seems rbd libraries are not thread-safe.

> 
> The simplest way would just be to detect rbd create with size 0 and
> print a clear error message like "image formats that grow on demand
> are not supported on rbd".

If we using this simplest way just give a clear error message, it means
following things:
1, Can create a qcow2 format image just using "-o preallocation=falloc or -o
preallocation=full".
2, Can not resize this qcow2 format image(created by above). In other words,
   even if we have resize this image successfully, can not write any data
   into the extend image.

I will submit a patch just using this simplest way.

Regards,
Jun Li