[Qemu-devel] [PULL 0/5] Block patches for 1.3.0-rc2

[Qemu-devel] [PULL 0/5] Block patches for 1.3.0-rc2

[Kevin Wolf]

The following changes since commit e9bff10f8db94912b1b0e6e2e3394cae02faf614:

  event notifier: Fix setup for win32 (2012-11-28 13:33:01 -0600)

are available in the git repository at:

  git://repo.or.cz/qemu/kevin.git for-anthony

Dietmar Maurer (1):
      stream: fix ratelimit_set_speed

Kevin Wolf (2):
      Documentation: Update block cache mode information
      Documentation: Update image format information

Pavel Hrdina (1):
      atapi: make change media detection for guests easier

Peter Maydell (1):
      coroutine-sigaltstack.c: Use stack_t, not struct sigaltstack

 coroutine-sigaltstack.c  |    4 +-
 hw/ide/atapi.c           |   15 +++--
 hw/ide/core.c            |    6 --
 include/qemu/ratelimit.h |    2 +-
 qemu-doc.texi            |  167 ++++++++++++++++++++++++++++++++++++++++++++++
 qemu-img.texi            |   84 ++++++++----------------
 qemu-options.hx          |   38 +++++-----
 7 files changed, 226 insertions(+), 90 deletions(-)

[Qemu-devel] [PATCH 1/5] Documentation: Update block cache mode information

[Kevin Wolf]

Somehow we forgot to update this when cache=writeback became the
default. While changing the information on the default, also make the
description of all caches modes a bit more accurate.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qemu-options.hx |   38 +++++++++++++++++++-------------------
 1 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/qemu-options.hx b/qemu-options.hx
index fbcf079..de43b1b 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -206,33 +206,33 @@ Open drive @option{file} as read-only. Guest write attempts will fail.
 file sectors into the image file.
 @end table
 
-By default, writethrough caching is used for all block device.  This means that
-the host page cache will be used to read and write data but write notification
-will be sent to the guest only when the data has been reported as written by
-the storage subsystem.
-
-Writeback caching will report data writes as completed as soon as the data is
-present in the host page cache.  This is safe as long as you trust your host.
-If your host crashes or loses power, then the guest may experience data
-corruption.
+By default, the @option{cache=writeback} mode is used. It will report data
+writes as completed as soon as the data is present in the host page cache.
+This is safe as long as your guest OS makes sure to correctly flush disk caches
+where needed. If your guest OS does not handle volatile disk write caches
+correctly and your host crashes or loses power, then the guest may experience
+data corruption.
+
+For such guests, you should consider using @option{cache=writethrough}. This
+means that the host page cache will be used to read and write data, but write
+notification will be sent to the guest only after QEMU has made sure to flush
+each write to the disk. Be aware that this has a major impact on performance.
 
 The host page cache can be avoided entirely with @option{cache=none}.  This will
-attempt to do disk IO directly to the guests memory.  QEMU may still perform
-an internal copy of the data.
+attempt to do disk IO directly to the guest's memory.  QEMU may still perform
+an internal copy of the data. Note that this is considered a writeback mode and
+the guest OS must handle the disk write cache correctly in order to avoid data
+corruption on host crashes.
 
 The host page cache can be avoided while only sending write notifications to
-the guest when the data has been reported as written by the storage subsystem
-using @option{cache=directsync}.
-
-Some block drivers perform badly with @option{cache=writethrough}, most notably,
-qcow2.  If performance is more important than correctness,
-@option{cache=writeback} should be used with qcow2.
+the guest when the data has been flushed to the disk using
+@option{cache=directsync}.
 
 In case you don't care about data integrity over host failures, use
-cache=unsafe. This option tells QEMU that it never needs to write any data
-to the disk but can instead keeps things in cache. If anything goes wrong,
+@option{cache=unsafe}. This option tells QEMU that it never needs to write any
+data to the disk but can instead keep things in cache. If anything goes wrong,
 like your host losing power, the disk storage getting disconnected accidentally,
-etc. you're image will most probably be rendered unusable.   When using
+etc. your image will most probably be rendered unusable.   When using
 the @option{-snapshot} option, unsafe caching is always used.
 
 Copy-on-read avoids accessing the same backing file sectors repeatedly and is
-- 
1.7.6.5

[Qemu-devel] [PATCH 2/5] Documentation: Update image format information

[Kevin Wolf]

Document new and yet undocumented options and image formats. The
qemu-img man page contains information only for raw and qcow2 now and
references the HTML documentation for a more detailed description of
other formats.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qemu-doc.texi |  167 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-img.texi |   84 +++++++++-------------------
 2 files changed, 194 insertions(+), 57 deletions(-)

diff --git a/qemu-doc.texi b/qemu-doc.texi
index 6ff309d..6d7f50d 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -416,6 +416,7 @@ snapshots.
 * vm_snapshots::              VM snapshots
 * qemu_img_invocation::       qemu-img Invocation
 * qemu_nbd_invocation::       qemu-nbd Invocation
+* disk_images_formats::       Disk image file formats
 * host_drives::               Using host drives
 * disk_images_fat_images::    Virtual FAT disk images
 * disk_images_nbd::           NBD access
@@ -507,6 +508,172 @@ state is not saved or restored properly (in particular USB).
 
 @include qemu-nbd.texi
 
+@node disk_images_formats
+@subsection Disk image file formats
+
+QEMU supports many image file formats that can be used with VMs as well as with
+any of the tools (like @code{qemu-img}). This includes the preferred formats
+raw and qcow2 as well as formats that are supported for compatibility with
+older QEMU versions or other hypervisors.
+
+Depending on the image format, different options can be passed to
+@code{qemu-img create} and @code{qemu-img convert} using the @code{-o} option.
+This section describes each format and the options that are supported for it.
+
+@table @option
+@item raw
+
+Raw disk image format. This format has the advantage of
+being simple and easily exportable to all other emulators. If your
+file system supports @emph{holes} (for example in ext2 or ext3 on
+Linux or NTFS on Windows), then only the written sectors will reserve
+space. Use @code{qemu-img info} to know the real size used by the
+image or @code{ls -ls} on Unix/Linux.
+
+@item qcow2
+QEMU image format, the most versatile format. Use it to have smaller
+images (useful if your filesystem does not supports holes, for example
+on Windows), optional AES encryption, zlib based compression and
+support of multiple VM snapshots.
+
+Supported options:
+@table @code
+@item compat
+Determines the qcow2 version to use. @code{compat=0.10} uses the traditional
+image format that can be read by any QEMU since 0.10 (this is the default).
+@code{compat=1.1} enables image format extensions that only QEMU 1.1 and
+newer understand. Amongst others, this includes zero clusters, which allow
+efficient copy-on-read for sparse images.
+
+@item backing_file
+File name of a base image (see @option{create} subcommand)
+@item backing_fmt
+Image format of the base image
+@item encryption
+If this option is set to @code{on}, the image is encrypted.
+
+Encryption uses the AES format which is very secure (128 bit keys). Use
+a long password (16 characters) to get maximum protection.
+
+@item cluster_size
+Changes the qcow2 cluster size (must be between 512 and 2M). Smaller cluster
+sizes can improve the image file size whereas larger cluster sizes generally
+provide better performance.
+
+@item preallocation
+Preallocation mode (allowed values: off, metadata). An image with preallocated
+metadata is initially larger but can improve performance when the image needs
+to grow.
+
+@item lazy_refcounts
+If this option is set to @code{on}, reference count updates are postponed with
+the goal of avoiding metadata I/O and improving performance. This is
+particularly interesting with @option{cache=writethrough} which doesn't batch
+metadata updates. The tradeoff is that after a host crash, the reference count
+tables must be rebuilt, i.e. on the next open an (automatic) @code{qemu-img
+check -r all} is required, which may take some time.
+
+This option can only be enabled if @code{compat=1.1} is specified.
+
+@end table
+
+@item qed
+Old QEMU image format with support for backing files and compact image files
+(when your filesystem or transport medium does not support holes).
+
+When converting QED images to qcow2, you might want to consider using the
+@code{lazy_refcounts=on} option to get a more QED-like behaviour.
+
+Supported options:
+@table @code
+@item backing_file
+File name of a base image (see @option{create} subcommand).
+@item backing_fmt
+Image file format of backing file (optional).  Useful if the format cannot be
+autodetected because it has no header, like some vhd/vpc files.
+@item cluster_size
+Changes the cluster size (must be power-of-2 between 4K and 64K). Smaller
+cluster sizes can improve the image file size whereas larger cluster sizes
+generally provide better performance.
+@item table_size
+Changes the number of clusters per L1/L2 table (must be power-of-2 between 1
+and 16).  There is normally no need to change this value but this option can be
+used for performance benchmarking.
+@end table
+
+@item qcow
+Old QEMU image format with support for backing files, compact image files,
+encryption and compression.
+
+Supported options:
+@table @code
+@item backing_file
+File name of a base image (see @option{create} subcommand)
+@item encryption
+If this option is set to @code{on}, the image is encrypted.
+@end table
+
+@item cow
+User Mode Linux Copy On Write image format. It is supported only for
+compatibility with previous versions.
+Supported options:
+@table @code
+@item backing_file
+File name of a base image (see @option{create} subcommand)
+@end table
+
+@item vdi
+VirtualBox 1.1 compatible image format.
+Supported options:
+@table @code
+@item static
+If this option is set to @code{on}, the image is created with metadata
+preallocation.
+@end table
+
+@item vmdk
+VMware 3 and 4 compatible image format.
+
+Supported options:
+@table @code
+@item backing_file
+File name of a base image (see @option{create} subcommand).
+@item compat6
+Create a VMDK version 6 image (instead of version 4)
+@item subformat
+Specifies which VMDK subformat to use. Valid options are
+@code{monolithicSparse} (default),
+@code{monolithicFlat},
+@code{twoGbMaxExtentSparse},
+@code{twoGbMaxExtentFlat} and
+@code{streamOptimized}.
+@end table
+
+@item vpc
+VirtualPC compatible image format (VHD).
+Supported options:
+@table @code
+@item subformat
+Specifies which VHD subformat to use. Valid options are
+@code{dynamic} (default) and @code{fixed}.
+@end table
+@end table
+
+@subsubsection Read-only formats
+More disk image file formats are supported in a read-only mode.
+@table @option
+@item bochs
+Bochs images of @code{growing} type.
+@item cloop
+Linux Compressed Loop image, useful only to reuse directly compressed
+CD-ROM images present for example in the Knoppix CD-ROMs.
+@item dmg
+Apple disk image.
+@item parallels
+Parallels disk image format.
+@end table
+
+
 @node host_drives
 @subsection Using host drives
 
diff --git a/qemu-img.texi b/qemu-img.texi
index 60b83fc..00fca8d 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -226,7 +226,10 @@ After using this command to grow a disk image, you must use file system and
 partitioning tools inside the VM to actually begin using the new space on the
 device.
 @end table
+@c man end
 
+@ignore
+@c man begin NOTES
 Supported image file formats:
 
 @table @option
@@ -247,6 +250,13 @@ support of multiple VM snapshots.
 
 Supported options:
 @table @code
+@item compat
+Determines the qcow2 version to use. @code{compat=0.10} uses the traditional
+image format that can be read by any QEMU since 0.10 (this is the default).
+@code{compat=1.1} enables image format extensions that only QEMU 1.1 and
+newer understand. Amongst others, this includes zero clusters, which allow
+efficient copy-on-read for sparse images.
+
 @item backing_file
 File name of a base image (see @option{create} subcommand)
 @item backing_fmt
@@ -267,73 +277,33 @@ Preallocation mode (allowed values: off, metadata). An image with preallocated
 metadata is initially larger but can improve performance when the image needs
 to grow.
 
-@end table
+@item lazy_refcounts
+If this option is set to @code{on}, reference count updates are postponed with
+the goal of avoiding metadata I/O and improving performance. This is
+particularly interesting with @option{cache=writethrough} which doesn't batch
+metadata updates. The tradeoff is that after a host crash, the reference count
+tables must be rebuilt, i.e. on the next open an (automatic) @code{qemu-img
+check -r all} is required, which may take some time.
 
-@item qed
-Image format with support for backing files and compact image files (when your
-filesystem or transport medium does not support holes).  Good performance due
-to less metadata than the more featureful qcow2 format, especially with
-cache=writethrough or cache=directsync.  Consider using qcow2 which will soon
-have a similar optimization and is most actively developed.
+This option can only be enabled if @code{compat=1.1} is specified.
 
-Supported options:
-@table @code
-@item backing_file
-File name of a base image (see @option{create} subcommand).
-@item backing_fmt
-Image file format of backing file (optional).  Useful if the format cannot be
-autodetected because it has no header, like some vhd/vpc files.
-@item cluster_size
-Changes the cluster size (must be power-of-2 between 4K and 64K). Smaller
-cluster sizes can improve the image file size whereas larger cluster sizes
-generally provide better performance.
-@item table_size
-Changes the number of clusters per L1/L2 table (must be power-of-2 between 1
-and 16).  There is normally no need to change this value but this option can be
-used for performance benchmarking.
 @end table
 
-@item qcow
-Old QEMU image format. Left for compatibility.
+@item Other
+QEMU also supports various other image file formats for compatibility with
+older QEMU versions or other hypervisors, including VMDK, VDI, VHD (vpc), qcow1
+and QED. For a full list of supported formats see @code{qemu-img --help}.
+For a more detailed description of these formats, see the QEMU Emulation User
+Documentation.
 
-Supported options:
-@table @code
-@item backing_file
-File name of a base image (see @option{create} subcommand)
-@item encryption
-If this option is set to @code{on}, the image is encrypted.
-@end table
-
-@item cow
-User Mode Linux Copy On Write image format. Used to be the only growable
-image format in QEMU. It is supported only for compatibility with
-previous versions. It does not work on win32.
-@item vdi
-VirtualBox 1.1 compatible image format.
-@item vmdk
-VMware 3 and 4 compatible image format.
-
-Supported options:
-@table @code
-@item backing_fmt
-Image format of the base image
-@item compat6
-Create a VMDK version 6 image (instead of version 4)
-@end table
-
-@item vpc
-VirtualPC compatible image format (VHD).
-
-@item cloop
-Linux Compressed Loop image, useful only to reuse directly compressed
-CD-ROM images present for example in the Knoppix CD-ROMs.
+The main purpose of the block drivers for these formats is image conversion.
+For running VMs, it is recommended to convert the disk images to either raw or
+qcow2 in order to achieve good performance.
 @end table
 
 
 @c man end
 
-@ignore
-
 @setfilename qemu-img
 @settitle QEMU disk image utility
 
-- 
1.7.6.5

[Qemu-devel] [PATCH 3/5] atapi: make change media detection for guests easier

[Kevin Wolf]

From: Pavel Hrdina <phrdina@redhat.com>

If you have a guest with a media in the optical drive and you change
it, the windows guest cannot properly recognize this media change.

Windows needs to detect sense "NOT_READY with ASC_MEDIUM_NOT_PRESENT"
before we send sense "UNIT_ATTENTION with ASC_MEDIUM_MAY_HAVE_CHANGED".

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 hw/ide/atapi.c |   15 ++++++++++-----
 hw/ide/core.c  |    6 ------
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index 685cbaa..861fd2b 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -1124,12 +1124,17 @@ void ide_atapi_cmd(IDEState *s)
      * GET_EVENT_STATUS_NOTIFICATION to detect such tray open/close
      * states rely on this behavior.
      */
-    if (!s->tray_open && bdrv_is_inserted(s->bs) && s->cdrom_changed) {
-        ide_atapi_cmd_error(s, NOT_READY, ASC_MEDIUM_NOT_PRESENT);
+    if (!(atapi_cmd_table[s->io_buffer[0]].flags & ALLOW_UA) &&
+        !s->tray_open && bdrv_is_inserted(s->bs) && s->cdrom_changed) {
+
+        if (s->cdrom_changed == 1) {
+            ide_atapi_cmd_error(s, NOT_READY, ASC_MEDIUM_NOT_PRESENT);
+            s->cdrom_changed = 2;
+        } else {
+            ide_atapi_cmd_error(s, UNIT_ATTENTION, ASC_MEDIUM_MAY_HAVE_CHANGED);
+            s->cdrom_changed = 0;
+        }
 
-        s->cdrom_changed = 0;
-        s->sense_key = UNIT_ATTENTION;
-        s->asc = ASC_MEDIUM_MAY_HAVE_CHANGED;
         return;
     }
 
diff --git a/hw/ide/core.c b/hw/ide/core.c
index 8da894f..c4f93d0 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -2160,12 +2160,6 @@ static int ide_drive_post_load(void *opaque, int version_id)
 {
     IDEState *s = opaque;
 
-    if (version_id < 3) {
-        if (s->sense_key == UNIT_ATTENTION &&
-            s->asc == ASC_MEDIUM_MAY_HAVE_CHANGED) {
-            s->cdrom_changed = 1;
-        }
-    }
     if (s->identify_set) {
         bdrv_set_enable_write_cache(s->bs, !!(s->identify_data[85] & (1 << 5)));
     }
-- 
1.7.6.5

[Qemu-devel] [PATCH 4/5] stream: fix ratelimit_set_speed

[Kevin Wolf]

From: Dietmar Maurer <dietmar@proxmox.com>

The formula to compute slice_quota was wrong since commit 6ef228fc.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 include/qemu/ratelimit.h |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/include/qemu/ratelimit.h b/include/qemu/ratelimit.h
index c6ac281..d1610f1 100644
--- a/include/qemu/ratelimit.h
+++ b/include/qemu/ratelimit.h
@@ -42,7 +42,7 @@ static inline void ratelimit_set_speed(RateLimit *limit, uint64_t speed,
                                        uint64_t slice_ns)
 {
     limit->slice_ns = slice_ns;
-    limit->slice_quota = ((double)speed * 1000000000ULL) / slice_ns;
+    limit->slice_quota = ((double)speed * slice_ns)/1000000000ULL;
 }
 
 #endif
-- 
1.7.6.5

[Qemu-devel] [PATCH 5/5] coroutine-sigaltstack.c: Use stack_t, not struct sigaltstack

[Kevin Wolf]

From: Peter Maydell <peter.maydell@linaro.org>

Use the POSIX-specified stack_t type as the argument to sigaltstack()
rather than the legacy struct sigaltstack. This allows us to compile
on MacOSX with --with-coroutine=sigaltstack.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 coroutine-sigaltstack.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/coroutine-sigaltstack.c b/coroutine-sigaltstack.c
index 861e878..39dbaa5 100644
--- a/coroutine-sigaltstack.c
+++ b/coroutine-sigaltstack.c
@@ -171,8 +171,8 @@ static Coroutine *coroutine_new(void)
     CoroutineThreadState *coTS;
     struct sigaction sa;
     struct sigaction osa;
-    struct sigaltstack ss;
-    struct sigaltstack oss;
+    stack_t ss;
+    stack_t oss;
     sigset_t sigs;
     sigset_t osigs;
     jmp_buf old_env;
-- 
1.7.6.5

Re: [Qemu-devel] [PULL 0/5] Block patches for 1.3.0-rc2

[Anthony Liguori]

Kevin Wolf <kwolf@redhat.com> writes:

> The following changes since commit e9bff10f8db94912b1b0e6e2e3394cae02faf614:
>
>   event notifier: Fix setup for win32 (2012-11-28 13:33:01 -0600)
>
> are available in the git repository at:
>
>   git://repo.or.cz/qemu/kevin.git for-anthony
>

Pulled. Thanks.

Regards,

Anthony Liguori

> Dietmar Maurer (1):
>       stream: fix ratelimit_set_speed
>
> Kevin Wolf (2):
>       Documentation: Update block cache mode information
>       Documentation: Update image format information
>
> Pavel Hrdina (1):
>       atapi: make change media detection for guests easier
>
> Peter Maydell (1):
>       coroutine-sigaltstack.c: Use stack_t, not struct sigaltstack
>
>  coroutine-sigaltstack.c  |    4 +-
>  hw/ide/atapi.c           |   15 +++--
>  hw/ide/core.c            |    6 --
>  include/qemu/ratelimit.h |    2 +-
>  qemu-doc.texi            |  167 ++++++++++++++++++++++++++++++++++++++++++++++
>  qemu-img.texi            |   84 ++++++++----------------
>  qemu-options.hx          |   38 +++++-----
>  7 files changed, 226 insertions(+), 90 deletions(-)