[Qemu-devel] Some uncertain about implementing stream optimized writing

[Qemu-devel] Some uncertain about implementing stream optimized writing

[Fam Zheng]

Hi,

I have implemented reading for sparse optimized and come to implement
writing. It is a little complicated and I am not sure what is the best
approach. Could you give me some advice?

Here is the details: (pasted from http://warm.la/soc/?p=98)

Stream optimized VMDK image allocates minimized space for a compressed
cluster, which means if there is high compress ratio, a cluster
possibly only takes one physical sector in the file. It makes
overwriting hard, especially when new data needs more sectors than
previously allocated.

Attach the image to VMware and boot the VM to test this format, it
seems that VMware wouldn’t do write to allocated clusters, but can
allocate new cluster to save data.

Overwriting existing cluster requires measuring new data size and
deciding whether in-place overwrite is OK, if not we must look for
other free space. Metadata in image has no such information for sector
allocation algorithm, so if we want to fully enable writing to stream
optimized, sector allocation bitmap will be introduced into block
state. This should significantly increase memory usage and somehow
complicate the driver.

Another approach I think of is to allocate each non-inplace at the end
of image and leave the old allocation unreferenced, which wastes disk
space.

-- 
Best regards!
Fam Zheng

Re: [Qemu-devel] Some uncertain about implementing stream optimized writing

[Stefan Hajnoczi]

On Wed, Jun 29, 2011 at 5:47 AM, Fam Zheng <famcool@gmail.com> wrote:
> Stream optimized VMDK image allocates minimized space for a compressed
> cluster, which means if there is high compress ratio, a cluster
> possibly only takes one physical sector in the file. It makes
> overwriting hard, especially when new data needs more sectors than
> previously allocated.
>
> Attach the image to VMware and boot the VM to test this format, it
> seems that VMware wouldn’t do write to allocated clusters, but can
> allocate new cluster to save data.

What happens when the VM writes to allocated clusters?  EIO?

Stefan

Re: [Qemu-devel] Some uncertain about implementing stream optimized writing

[Fam Zheng]

No error report with writing (dd of=/dev/sda ...), but can't change
data, (dd if=/dev/sda) dump unchanged.

On Wed, Jun 29, 2011 at 1:47 PM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Wed, Jun 29, 2011 at 5:47 AM, Fam Zheng <famcool@gmail.com> wrote:
>> Stream optimized VMDK image allocates minimized space for a compressed
>> cluster, which means if there is high compress ratio, a cluster
>> possibly only takes one physical sector in the file. It makes
>> overwriting hard, especially when new data needs more sectors than
>> previously allocated.
>>
>> Attach the image to VMware and boot the VM to test this format, it
>> seems that VMware wouldn’t do write to allocated clusters, but can
>> allocate new cluster to save data.
>
> What happens when the VM writes to allocated clusters?  EIO?
>
> Stefan
>



-- 
Best regards!
Fam Zheng

Re: [Qemu-devel] Some uncertain about implementing stream optimized writing

[Stefan Hajnoczi]

On Wed, Jun 29, 2011 at 7:18 AM, Fam Zheng <famcool@gmail.com> wrote:
> No error report with writing (dd of=/dev/sda ...), but can't change
> data, (dd if=/dev/sda) dump unchanged.

I suggest we return EIO on non-allocating writes.  That way the user
will know there is a problem with what they are trying to do.
qemu-img convert will work fine but running read-write guests will
error.

Stefan

Re: [Qemu-devel] Some uncertain about implementing stream optimized writing

[Kevin Wolf]

Am 29.06.2011 06:47, schrieb Fam Zheng:
> Hi,
> 
> I have implemented reading for sparse optimized and come to implement
> writing. It is a little complicated and I am not sure what is the best
> approach. Could you give me some advice?
> 
> Here is the details: (pasted from http://warm.la/soc/?p=98)
> 
> Stream optimized VMDK image allocates minimized space for a compressed
> cluster, which means if there is high compress ratio, a cluster
> possibly only takes one physical sector in the file. It makes
> overwriting hard, especially when new data needs more sectors than
> previously allocated.
> 
> Attach the image to VMware and boot the VM to test this format, it
> seems that VMware wouldn’t do write to allocated clusters, but can
> allocate new cluster to save data.

I would have expected that they copy the data to a new cluster, like
qcow2 does.

Of course, it doesn't make a lot of sense to do COW if you can't free
the old space. Doing qcow2 style compression (which it seems to be from
your description) without having reference counts seems a bit stupid...

If you really can't write to already allocated clusters in VMware,
what's the use case for this? Have one read-only partition and
everything else free?

> Overwriting existing cluster requires measuring new data size and
> deciding whether in-place overwrite is OK, if not we must look for
> other free space. Metadata in image has no such information for sector
> allocation algorithm, so if we want to fully enable writing to stream
> optimized, sector allocation bitmap will be introduced into block
> state. This should significantly increase memory usage and somehow
> complicate the driver.

That would be 256k per GB. Sounds like it could become very large
indeed. Maybe better a free list approach and an image scan, like what
was considered for QED. Such a scan is unacceptable for a native format,
but maybe good enough for VMDK.

But how much is this format really used? Is it worth the trouble? If it
doesn't exist in practice, let's just leak the clusters. Whoever needs
more disk space should use qemu-img convert to compact it again.

Kevin

Re: [Qemu-devel] Some uncertain about implementing stream optimized writing

[Fam Zheng]

On Wed, Jun 29, 2011 at 4:15 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 29.06.2011 06:47, schrieb Fam Zheng:
>> Hi,
>>
>> I have implemented reading for sparse optimized and come to implement
>> writing. It is a little complicated and I am not sure what is the best
>> approach. Could you give me some advice?
>>
>> Here is the details: (pasted from http://warm.la/soc/?p=98)
>>
>> Stream optimized VMDK image allocates minimized space for a compressed
>> cluster, which means if there is high compress ratio, a cluster
>> possibly only takes one physical sector in the file. It makes
>> overwriting hard, especially when new data needs more sectors than
>> previously allocated.
>>
>> Attach the image to VMware and boot the VM to test this format, it
>> seems that VMware wouldn’t do write to allocated clusters, but can
>> allocate new cluster to save data.
>
> I would have expected that they copy the data to a new cluster, like
> qcow2 does.
>
> Of course, it doesn't make a lot of sense to do COW if you can't free
> the old space. Doing qcow2 style compression (which it seems to be from
> your description) without having reference counts seems a bit stupid...
>
> If you really can't write to already allocated clusters in VMware,
> what's the use case for this? Have one read-only partition and
> everything else free?

I think it's only for improving transmission efficiency, as the name
"stream optimized" indicates.

>
>> Overwriting existing cluster requires measuring new data size and
>> deciding whether in-place overwrite is OK, if not we must look for
>> other free space. Metadata in image has no such information for sector
>> allocation algorithm, so if we want to fully enable writing to stream
>> optimized, sector allocation bitmap will be introduced into block
>> state. This should significantly increase memory usage and somehow
>> complicate the driver.
>
> That would be 256k per GB. Sounds like it could become very large
> indeed. Maybe better a free list approach and an image scan, like what
> was considered for QED. Such a scan is unacceptable for a native format,
> but maybe good enough for VMDK.
>
> But how much is this format really used? Is it worth the trouble? If it
> doesn't exist in practice, let's just leak the clusters. Whoever needs
> more disk space should use qemu-img convert to compact it again.
>
It does exist and vmware-vdiskmanager produces such format. But I
don't think in practice anyone would try to run a VM on it.


-- 
Best regards!
Fam Zheng