All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Richard W.M. Jones" <rjones@redhat.com>
To: Martin Kletzander <mkletzan@redhat.com>
Cc: qemu-devel@nongnu.org, Kevin Wolf <kwolf@redhat.com>,
	Eric Blake <eblake@redhat.com>
Subject: Re: [Qemu-devel] Possibly incorrect data sparsification by qemu-img
Date: Tue, 23 Apr 2019 12:36:02 +0100	[thread overview]
Message-ID: <20190423113602.GJ3926@redhat.com> (raw)
In-Reply-To: <20190423113028.GD30014@wheatley>

On Tue, Apr 23, 2019 at 01:30:28PM +0200, Martin Kletzander wrote:
> I am using qemu-img with nbdkit to transfer a disk image and the
> update it with extra data from newer snapshots.  The end image
> cannot be transferred because the snapshots will be created later
> than the first transfer and we want to save some time up front.  You
> might think of it as a continuous synchronisation.

It's important to note here that Martin is reading the data from a
VMware server, so this is not something that can be solved with qemu's
own snapshots.

[...]

I think the following is an even simpler demo which gets to the nub of
the problem as I understand it:

$ rm -f disk.img snap.img
$ dd if=/dev/urandom of=disk.img bs=2M count=1
$ dd if=/dev/zero of=snap.img bs=2M count=1
$ qemu-img convert -n -p snap.img disk.img
$ hexdump -C disk.img | head
00000000  18 30 e8 1f 09 f0 bb 2c  2f c7 b3 97 8f 12 fe 4b  |.0.....,/......K|
00000010  66 f7 28 cb 8e 72 2a 37  6b fa 98 2e a0 e6 d9 cf  |f.(..r*7k.......|
[etc] <- ie. not zeroes

Should we expect disk.img to contain zeroes at the end?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

WARNING: multiple messages have this Message-ID (diff)
From: "Richard W.M. Jones" <rjones@redhat.com>
To: Martin Kletzander <mkletzan@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>, qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] Possibly incorrect data sparsification by qemu-img
Date: Tue, 23 Apr 2019 12:36:02 +0100	[thread overview]
Message-ID: <20190423113602.GJ3926@redhat.com> (raw)
Message-ID: <20190423113602.zwyqzctYX2kEede5HmmBJ9-Kz8xs56M_XALCXTX1mqQ@z> (raw)
In-Reply-To: <20190423113028.GD30014@wheatley>

On Tue, Apr 23, 2019 at 01:30:28PM +0200, Martin Kletzander wrote:
> I am using qemu-img with nbdkit to transfer a disk image and the
> update it with extra data from newer snapshots.  The end image
> cannot be transferred because the snapshots will be created later
> than the first transfer and we want to save some time up front.  You
> might think of it as a continuous synchronisation.

It's important to note here that Martin is reading the data from a
VMware server, so this is not something that can be solved with qemu's
own snapshots.

[...]

I think the following is an even simpler demo which gets to the nub of
the problem as I understand it:

$ rm -f disk.img snap.img
$ dd if=/dev/urandom of=disk.img bs=2M count=1
$ dd if=/dev/zero of=snap.img bs=2M count=1
$ qemu-img convert -n -p snap.img disk.img
$ hexdump -C disk.img | head
00000000  18 30 e8 1f 09 f0 bb 2c  2f c7 b3 97 8f 12 fe 4b  |.0.....,/......K|
00000010  66 f7 28 cb 8e 72 2a 37  6b fa 98 2e a0 e6 d9 cf  |f.(..r*7k.......|
[etc] <- ie. not zeroes

Should we expect disk.img to contain zeroes at the end?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW


  reply	other threads:[~2019-04-23 11:36 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-23 11:30 [Qemu-devel] Possibly incorrect data sparsification by qemu-img Martin Kletzander
2019-04-23 11:30 ` Martin Kletzander
2019-04-23 11:36 ` Richard W.M. Jones [this message]
2019-04-23 11:36   ` Richard W.M. Jones
2019-04-23 11:55   ` Daniel P. Berrangé
2019-04-23 12:12 ` Kevin Wolf
2019-04-23 12:12   ` Kevin Wolf
2019-04-23 14:26   ` Martin Kletzander
2019-04-23 14:26     ` Martin Kletzander
2019-04-23 15:08     ` Kevin Wolf
2019-04-23 15:08       ` Kevin Wolf
2019-04-24  6:40       ` Vladimir Sementsov-Ogievskiy
2019-04-24  7:19         ` Kevin Wolf
2019-04-24  9:04           ` Martin Kletzander
2019-04-29  7:27           ` Martin Kletzander
2019-04-29  8:58             ` Vladimir Sementsov-Ogievskiy
2019-04-29  9:16               ` Martin Kletzander

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190423113602.GJ3926@redhat.com \
    --to=rjones@redhat.com \
    --cc=eblake@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=mkletzan@redhat.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.